feedforward neural network
In plain wordsA neural network laid out like an assembly line: each stage hands its result only to later stages, never back, so every answer depends on the current input alone and nothing is remembered between inputs.
A neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. whose connection graph contains no cycles (a directed acyclic graph), so that each unit's value depends only on units earlier in a fixed ordering and the whole network computes a fixed function of the current input in one pass, keeping no state from one input to the next. The multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model. and the convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. are feedforward, and so, within a single forward passforward pass, n.: The step where a neural network takes an input and runs it through its layers, one after another, to produce an output — its answer or prediction., is the transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.; recurrent networksrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood., whose activations feed back into themselves across time steps, are the main contrast. Because the computation graph is acyclic, exact gradients can be obtained by one backward sweep of backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction.. Depth is the length of the longest path from input to output.
A feedforward classifier treats every frame independently, so it cannot use what happened a second earlier.
Origin
The feedforward/feedback distinction comes from control engineering and neuroscience. Frank Rosenblatt's perceptron work (1958–1962) separated 'series-coupled' networks, in which signals pass strictly forward, from cross-coupled and back-coupled ones. 'Feedforward network' became the standard connectionist term in the 1980s. Linear regression fitted by least squares (Legendre, Gauss) is sometimes described as the simplest feedforward network.
Usage note
Not to be confused with the feed-forward networkfeed-forward network, n.: The part of each transformer layer that works on every word separately, after attention has mixed in context. It holds most of a model's adjustable numbers and is thought to store much of its knowledge. sublayer of a transformer block, a small MLP applied independently at each token position. Autoregressive generation feeds outputs back as inputs, but the network itself remains feedforward.
Related
- multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.
- forward passforward pass, n.: The step where a neural network takes an input and runs it through its layers, one after another, to produce an output — its answer or prediction.
- computational graphcomputational graph, n.: A map of a calculation drawn as boxes and arrows: each box is a simple operation and each arrow carries a number to the next step. AI software uses it to trace errors backward.
- layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage.
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.
- recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood.
Mentioned in
neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., backpropagation through timebackpropagation through time, n.: The way recurrent networks are trained: the network is unrolled into a long chain, one copy per time step, and errors are traced back through every step to adjust the shared weights., feed-forward networkfeed-forward network, n.: The part of each transformer layer that works on every word separately, after attention has mixed in context. It holds most of a model's adjustable numbers and is thought to store much of its knowledge., multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model., recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood., highway networkhighway network, n.: An early design for very deep networks in which learned gates let information either pass through a layer unchanged or be transformed, making networks hundreds of layers deep trainable., radial basis function networkradial basis function network, n.: A three-layer network whose hidden neurons each respond to inputs near a particular reference point, like bells placed across the data; the output is a weighted mix of the bells., recurrent processing theoryrecurrent processing theory, n.: A theory that we consciously see something only when signals in the brain's visual areas loop back on themselves, not when they merely sweep forward once., time delay neural networktime delay neural network, n.: An early network for speech that applied the same small pattern detector at every moment of a sound, so it could recognize a sound wherever it occurred in time.