Skip to content
Last Dictionary
Plane
Deep LearningVision, Speech & Perception

pooling

nounAlso: max pooling, average pooling, global average pooling, pooling layer, subsampling, mean pooling

In plain wordsA step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions.

  1. 1

    Convolutional networks

    A parameter-free layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage. operation that reduces the spatial size of feature mapsfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears. by summarizing each local window with one value, the maximum (max pooling) or the mean (average pooling), separately for each channel. A typical 2 × 2 window with stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result. 2 halves height and width and discards three-quarters of the activations. Pooling cuts computation, enlarges the receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to. of later layers and makes outputs locally invariantinvariance, n.: A property of a model whose answer does not change when the input is altered in an irrelevant way, such as a photo classifier that still says 'cat' when the cat moves or the picture is flipped. to small shifts. Global average pooling, which averages each channel over the whole map, replaced large fully connectedfully connected layer, n.: A layer in which every input is connected to every output by its own adjustable weight. It is the most general and most common kind of neural-network layer. heads in classifiers such as GoogLeNet and ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI.. Many modern networks downsample with strided convolutions instead, and capsule networkcapsule network, n.: A network design championed by Geoffrey Hinton in which groups of neurons, called capsules, each describe a part of an object and its position, so the network can check whether the parts fit together. proponents criticized pooling for discarding spatial relationships between parts.

    Each 2 × 2 max pooling layer halves the resolution of the feature maps.

  2. 2

    Sequences and graphs

    Any operation that aggregates a variable number of vectors into one fixed-size vector, such as mean-pooling token embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. into a sentence embedding, taking a designated classification token's state, or a readout over node states in a graph neural networkgraph neural network, n.: A neural network for data shaped like a web of connections, such as molecules, social networks or road maps, in which each point repeatedly gathers information from its neighbors..

Origin

Kunihiko Fukushima's neocognitron (1980) used fixed downsampling 'C-cell' layers. Max pooling was introduced by Kouichi Yamaguchi and colleagues in 1990 for speech recognition and brought into vision by Juyang Weng and colleagues' Cresceptron (1992–93); it was standard in CNNs of the AlexNet era.

Related

See also
  • convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
  • feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
  • receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to.
  • stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
  • invarianceinvariance, n.: A property of a model whose answer does not change when the input is altered in an irrelevant way, such as a photo classifier that still says 'cat' when the cat moves or the picture is flipped.
  • capsule networkcapsule network, n.: A network design championed by Geoffrey Hinton in which groups of neurons, called capsules, each describe a part of an object and its position, so the network can check whether the parts fit together.

Mentioned in

layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears., receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to., ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI., stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result., U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost., capsule networkcapsule network, n.: A network design championed by Geoffrey Hinton in which groups of neurons, called capsules, each describe a part of an object and its position, so the network can check whether the parts fit together., fully convolutional networkfully convolutional network, n.: An image network made only of scanning-window layers, with no fixed-size final layers, so it can take pictures of any size and label every pixel., InceptionInception, prop. n.: A family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low., invarianceinvariance, n.: A property of a model whose answer does not change when the input is altered in an irrelevant way, such as a photo classifier that still says 'cat' when the cat moves or the picture is flipped., Kunihiko FukushimaKunihiko Fukushima, prop. n.: A Japanese engineer who in 1979 invented the neocognitron, a brain-inspired image-recognition network that was the direct ancestor of today's convolutional neural networks., squeeze-and-excitation networksqueeze-and-excitation network, n.: An image-network design whose add-on blocks let the network decide, for each picture, which of its internal feature channels to turn up and which to turn down., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.