convolution
In plain wordsAn operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
- 1
Deep learning
A layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage. operation that computes each output value as a weighted sum of a small local window of the input, using the same weights (the filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture. or kernel) at every position. For an input with C_in channels, a 2-D convolutional layer with C_out filters of size k × k has C_out·C_in·k² weights plus biases, independent of image size, and produces C_out output channels or feature mapsfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.. Its hyperparameters are kernel size, stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result., padding (border values, usually zeros, that control output size), dilationdilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost. and groups. 1-D convolutions process sequences such as audio or text; 3-D convolutions process video and volumetric scans. Variants include depthwise separabledepthwise separable convolution, n.: A cheaper way to do the main operation of image networks: first scan each channel of the input separately, then mix the channels together with a tiny one-by-one step., transposedtransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image. and 1 × 1 (pointwise) convolutions.
The stem is a single 7 × 7 convolution with stride 2.
- 2
Mathematics
The operation (f ∗ g)(t) = ∫ f(τ) g(t − τ) dτ, or its discrete sum, which flips one function and slides it across the other. It commutes with translation, and by the convolution theorem it becomes pointwise multiplication under the Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors.. The deep-learning operation omits the flip, so it is strictly cross-correlation.
Origin
The operation is classical in analysis, probability and signal processing, where linear filters are applied by convolution. In neural networks, Toshiteru Homma, Les Atlas and Robert Marks used the term at the first NIPS conference (1987), noting that convolution and correlation are equivalent when the weights are learned; it became standard through Yann LeCun's convolutional networks of 1989 onwards.
Usage note
Practitioners say 'kernel' both for filter weights and for low-level GPU routines; see kernelkernel, n.: A word with several AI meanings: a similarity measure used by some learning methods; a small grid of numbers that scans an image inside a neural network; or a small program that runs on a GPU..
Related
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture.
- feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
- stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
- depthwise separable convolutiondepthwise separable convolution, n.: A cheaper way to do the main operation of image networks: first scan each channel of the input separately, then mix the channels together with a tiny one-by-one step.
- transposed convolutiontransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image.
- dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost.
- Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors.
Mentioned in
layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears., filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture., fully connected layerfully connected layer, n.: A layer in which every input is connected to every output by its own adjustable weight. It is the most general and most common kind of neural-network layer., kernelkernel, n.: A word with several AI meanings: a similarity measure used by some learning methods; a small grid of numbers that scans an image inside a neural network; or a small program that runs on a GPU., point cloudpoint cloud, n.: A 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos., ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI., stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result., U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost., WaveNetWaveNet, prop. n.: A DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech., ConformerConformer, prop. n.: A neural-network design for speech recognition that combines a transformer, which relates distant parts of a recording, with convolutions, which pick up fine local sound patterns., depthwise separable convolutiondepthwise separable convolution, n.: A cheaper way to do the main operation of image networks: first scan each channel of the input separately, then mix the channels together with a tiny one-by-one step., dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost., equivarianceequivariance, n.: A property of a model whose output changes in step with its input: move the cat in a photo, and the network's internal 'cat' signal moves the same way., Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors., fully convolutional networkfully convolutional network, n.: An image network made only of scanning-window layers, with no fixed-size final layers, so it can take pictures of any size and label every pixel., GPGPUGPGPU, abbr.: Using a graphics chip for ordinary number-crunching instead of drawing pictures — the trick that, years later, made modern AI training practical., InceptionInception, prop. n.: A family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low., neural operatorneural operator, n.: A neural network that learns how a whole physical system responds, such as how a fluid flows given its starting state, so it can stand in for slow simulations and answer new cases quickly., patch embeddingpatch embedding, n.: The first step of a vision transformer: cutting an image into small square tiles and turning each tile into a list of numbers that the model can process like a word., systolic arraysystolic array, n.: A grid of tiny calculators that pass numbers to their neighbors in rhythm, like a heartbeat, so each value is reused many times. AI chips use it to multiply large tables of numbers efficiently., transposed convolutiontransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.