receptive field
In plain wordsThe patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to.
For a unit in a convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. or other locally connected network, the region of the input that can influence its value. A unit in a first convolutional layer with a 3 × 3 filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture. has a 3 × 3 receptive field; stacking layers, poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions., stridingstride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result. and dilationdilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost. enlarge it, so deep units can respond to whole objects. The theoretical receptive field grows linearly with depth for stride-1 convolutions and much faster with downsampling or exponentially increasing dilation. The effective receptive field, the part that actually matters, is smaller: Wenjie Luo and colleagues (2016) found its influence roughly Gaussian and confined to a fraction of the theoretical field. Matching receptive-field size to the scale of the relevant patterns is a design concern in segmentation, detection and audio models.
Our units' receptive field is only 100 pixels, too small to see the whole lesion.
Origin
Coined by physiologist Charles Sherrington in 1906 for the patch of skin from which a dog's scratch reflex could be triggered. Haldan Keffer Hartline applied it to single retinal neurons in 1938, and David Hubel and Torsten Wiesel's studies of cat visual cortex (1959–1962) inspired the local connectivity of Kunihiko Fukushima's neocognitron (1980) and later CNNs.
Related
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions.
- dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost.
- feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
- stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
- filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture.
Sources
- Luo, W., Li, Y., Urtasun, R. & Zemel, R. (2016). Understanding the Effective Receptive Field in Deep Convolutional Neural Networks. NIPS 2016; arXiv:1701.04128. arxiv.org/abs/1701.04128
Mentioned in
convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears., filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture., kernelkernel, n.: A word with several AI meanings: a similarity measure used by some learning methods; a small grid of numbers that scans an image inside a neural network; or a small program that runs on a GPU., poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions., stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result., WaveNetWaveNet, prop. n.: A DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech., dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost., Kunihiko FukushimaKunihiko Fukushima, prop. n.: A Japanese engineer who in 1979 invented the neocognitron, a brain-inspired image-recognition network that was the direct ancestor of today's convolutional neural networks., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.