neural network
In plain wordsA kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain.
- 1
Machine learning
A modelmodel, n.: A simplified working version of something. In AI it usually means the trained program, a big set of learned numbers, that takes an input and produces an answer, prediction or piece of content. that builds a complicated function out of many instances of one simple operation, the artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output., which takes a weighted sum of its inputs, adds a biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system. and applies a nonlinear activation functionactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer.. Units are usually organized in layerslayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., each feeding the next, and the connection strengths, or weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself., are parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. fitted to data, usually by backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction. and gradient descentgradient descent, n.: A method for training models by repeatedly nudging every setting a small step in whichever direction most reduces the error, like a hiker who cannot see the valley but keeps stepping wherever the ground tilts down. on a loss functionloss function, n.: A score that measures how wrong a model's predictions are. Training means adjusting the model, step by step, to push this number down.. With enough units, networks can approximate a very wide class of functions (the universal approximation theoremuniversal approximation theorem, n.: A mathematical result showing that a neural network with enough neurons can imitate essentially any smooth input-to-output relationship as closely as you like, though it does not say how to find that network.). Architectures differ in how units connect: feedforwardfeedforward neural network, n.: A neural network laid out like an assembly line: each stage hands its result only to later stages, never back, so every answer depends on the current input alone and nothing is remembered between inputs. networks such as the multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.; convolutionalconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. networks for grids such as images; recurrentrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood. networks for sequences; graphgraph neural network, n.: A neural network for data shaped like a web of connections, such as molecules, social networks or road maps, in which each point repeatedly gathers information from its neighbors. networks; and transformerstransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it., which underlie large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.. Training networks with many layers is the subject of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract..
The neural network learned to flag weld defects that the inspectors' checklist missed.
- 2
Neuroscience
A population of biological neurons connected by synapses, such as a circuit in the brain. Artificial networks borrowed the vocabulary of neurons, synapses and firing from neuroscience, but they are drastically simplified and are trained by methods, such as backpropagation, with no established counterpart in brains.
Origin
Warren McCulloch and Walter Pitts's 1943 paper modeled neurons as logical threshold units connected in 'nets'; Frank Rosenblatt's perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong. (1958) added learning. After the critique in PerceptronsPerceptrons (book), prop. n.: A 1969 book by Marvin Minsky and Seymour Papert proving that simple one-layer neural networks cannot learn certain patterns. It is widely blamed for a decade-long slump in neural-network research. (1969), the approach revived in the 1980s with backpropagation and the connectionistconnectionism, n.: The view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols. movement, when 'artificial neural network' became standard, and again after 2006 under the banner of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract..
Usage note
The brain analogy is loose: artificial neurons are arithmetic operations, not cells, and talk of a network 'thinking' is metaphor. 'Neural network' names the model family; deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract. names the practice of training deep ones.
Related
- artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output.
- deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.
- perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong.
- backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction.
- McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
- connectionismconnectionism, n.: The view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols.
- weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself.
- symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data.
- decision treedecision tree, n.: A model that makes a prediction by asking a series of yes-or-no questions about the input, like a flowchart, with each answer leading to the next question until it reaches a final answer.
Sources
- McCulloch, W. S. & Pitts, W. (1943). A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics, 5, 115–133.
- Rumelhart, D. E., Hinton, G. E. & Williams, R. J. (1986). Learning Representations by Back-propagating Errors. Nature, 323, 533–536.
Mentioned in
AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models., artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output., deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract., Geoffrey HintonGeoffrey Hinton, prop. n.: A British-Canadian scientist whose decades of persistent work on brain-inspired neural networks laid the foundations of modern AI; often called a 'godfather of AI', he now warns publicly about its dangers., layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats., mechanistic interpretabilitymechanistic interpretability, n.: A branch of AI research that tries to reverse-engineer a neural network — working out, piece by piece, which internal parts represent which concepts and how they combine to produce behavior., modelmodel, n.: A simplified working version of something. In AI it usually means the trained program, a big set of learned numbers, that takes an input and produces an answer, prediction or piece of content., transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it., activation steeringactivation steering, n.: Changing an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model., architecturearchitecture, n.: The overall design of an AI model, meaning what kinds of parts it has and how they are connected, as opposed to the specific numbers it learns during training., autoencoderautoencoder, n.: A neural network trained to squeeze its input into a compact code and then rebuild the original from that code. Learning to compress this way forces it to capture what matters in the data., Chinese roomChinese room, n.: A thought experiment: a person who knows no Chinese follows a rulebook to answer Chinese messages so well that outsiders think they understand. If the person doesn't understand, does a computer running a program?, cognitive sciencecognitive science, n.: The scientific study of how minds work — how we reason, use language, remember and see — combining psychology, neuroscience, linguistics, philosophy, anthropology and artificial intelligence., computational theory of mindcomputational theory of mind, n.: The idea that thinking is a kind of computation: the mind processes information by following rules, much as a computer runs a program, and the brain is the machine doing it., connectionismconnectionism, n.: The view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols., convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., Dartmouth workshopDartmouth workshop, prop. n.: A summer meeting of scientists at Dartmouth College in 1956, planned in a 1955 funding proposal that first used the phrase 'artificial intelligence'. Most histories treat it as the moment AI became a research field of its own., David RumelhartDavid Rumelhart, prop. n.: An American psychologist who, with Geoffrey Hinton and Ronald Williams, showed in 1986 how to train multilayer neural networks, launching the modern revival of the field., deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images., Donald HebbDonald Hebb, prop. n.: A Canadian psychologist who proposed in 1949 that connections between brain cells strengthen when the cells are active together, an idea behind many learning rules for neural networks., end-to-end learningend-to-end learning, n.: Training one AI system to go straight from raw input, such as camera images, to the final output, such as steering commands, instead of chaining hand-designed steps., feedforward neural networkfeedforward neural network, n.: A neural network laid out like an assembly line: each stage hands its result only to later stages, never back, so every answer depends on the current input alone and nothing is remembered between inputs., formal verificationformal verification, n.: Using mathematics, not just testing, to prove that a computer system always behaves as specified — for example, that a chip never adds wrongly or that a program can never crash in a particular way.