Inception
In plain wordsA family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low.
A family of convolutional networksconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. built from Inception modules: blocks that apply 1 × 1, 3 × 3 and 5 × 5 convolutionsconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location. and a max-poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions. branch in parallel to the same input and concatenate the results, so each layer captures patterns at several scales. Cheap 1 × 1 convolutions first shrink the number of channels entering the costly larger filters, which, as its designers put it, let the network grow deeper and wider within a fixed computational budget. The first version, GoogLeNet, was 22 layers deep, used auxiliary classifiers on intermediate layers to aid training and global average pooling instead of large fully connected layersfully connected layer, n.: A layer in which every input is connected to every output by its own adjustable weight. It is the most general and most common kind of neural-network layer., and had about 12 times fewer parameters than AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.. Later versions added batch normalizationbatch normalization, n.: A layer that rescales the numbers flowing through a network so that, across each batch of training examples, they keep a steady average and spread. It makes deep networks train faster and more reliably. and factorized convolutions (Inception-v2 and v3, 2015); Inception-v4 (2016) refined the modules, and its sibling Inception-ResNet added residual connectionsresidual connection, n.: A shortcut in a neural network that adds a block's input straight to its output, so each block only has to learn a correction. It is what makes networks hundreds of layers deep trainable.. Features from Inception-v3 underlie the Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better. used to evaluate image generators.
FID is computed on Inception-v3 pool features, so it inherits that network's biases.
Origin
Introduced by Christian Szegedy and eight co-authors, most of them at Google, in 'Going Deeper with Convolutions' (arXiv, September 2014; CVPR 2015). GoogLeNet, named in homage to LeNet, won the ILSVRC 2014 classification task with about 6.7% top-5 error. The name Inception nods to the internet meme 'we need to go deeper', from the film Inception, which the paper cites.
Related
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.
- ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI.
- AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.
- Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better.
- depthwise separable convolutiondepthwise separable convolution, n.: A cheaper way to do the main operation of image networks: first scan each channel of the input separately, then mix the channels together with a tiny one-by-one step.
Sources
- Szegedy, C. et al. (2014). Going Deeper with Convolutions. arXiv:1409.4842; CVPR 2015. arxiv.org/abs/1409.4842
Mentioned in
squeeze-and-excitation networksqueeze-and-excitation network, n.: An image-network design whose add-on blocks let the network decide, for each picture, which of its internal feature channels to turn up and which to turn down., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.