connectionism
In plain wordsThe view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols.
An approach to cognitive science and artificial intelligence that models mental processes as the emergent activity of networks of simple interconnected units, with knowledge stored in the strengths of the connections and acquired by learning rather than programmed as explicit rules. Connectionist models emphasize parallel processing, distributed representations (each concept spread across many units, each unit shared among many concepts), graceful degradation under damage and generalization from examples. Rooted in the McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic., Hebbian learningHebbian learning, n.: The idea that a connection between two brain cells grows stronger when they are active at the same time — the principle behind the slogan 'cells that fire together wire together'. and the perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong., the approach revived in the 1980s with the Hopfield networkHopfield network, n.: A network of on–off units, all connected to one another, that stores patterns as stable states. Given a partial or noisy pattern, it settles into the closest stored one, like recalling a memory from a hint., the Boltzmann machineBoltzmann machine, n.: A network of randomly flipping on–off units, including hidden ones, that learns the statistical patterns in its training data and can then produce new examples like them. and backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction., and is the intellectual ancestor of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.. For decades it stood opposed to symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data. in a debate over whether cognition requires structured manipulation of symbols.
A connectionist model treats the English past tense as a pattern in learned weights rather than as an explicit rule plus exceptions.
Origin
Edward Thorndike had called his early-twentieth-century stimulus–response theory of learning connectionism. The modern sense was set by Jerome Feldman and Dana Ballard's 'Connectionist Models and Their Properties' (Cognitive Science, 1982) and spread with the Parallel Distributed ProcessingParallel Distributed Processing, prop. n.: A two-volume 1986 book by a group of psychologists and computer scientists that revived interest in brain-like neural networks and made the training method behind modern AI widely known. volumes of 1986.
Usage note
Jerry Fodor and Zenon Pylyshyn's 1988 critique argued that connectionist networks cannot explain the systematic structure of thought; the argument continues in debates over whether large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. need symbolic components (see neuro-symbolic AIneuro-symbolic AI, n.: AI that combines neural networks, which learn patterns from data, with symbolic methods, which follow explicit rules and logic, aiming for systems that both learn flexibly and reason reliably.).
Related
- Parallel Distributed ProcessingParallel Distributed Processing, prop. n.: A two-volume 1986 book by a group of psychologists and computer scientists that revived interest in brain-like neural networks and made the training method behind modern AI widely known.
- neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain.
- deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.
- symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data.
- neuro-symbolic AIneuro-symbolic AI, n.: AI that combines neural networks, which learn patterns from data, with symbolic methods, which follow explicit rules and logic, aiming for systems that both learn flexibly and reason reliably.
- neats and scruffiesneats and scruffies, phr.: A joking split of AI researchers into 'neats', who want clean mathematical theories that can be proved correct, and 'scruffies', who build messy collections of tricks that simply work.
- symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data.
Sources
- Feldman, J. A. & Ballard, D. H. (1982). Connectionist Models and Their Properties. Cognitive Science 6(3), 205–254.
- Fodor, J. A. & Pylyshyn, Z. W. (1988). Connectionism and Cognitive Architecture: A Critical Analysis. Cognition 28(1–2), 3–71.
Mentioned in
neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data., cognitive sciencecognitive science, n.: The scientific study of how minds work — how we reason, use language, remember and see — combining psychology, neuroscience, linguistics, philosophy, anthropology and artificial intelligence., computational theory of mindcomputational theory of mind, n.: The idea that thinking is a kind of computation: the mind processes information by following rules, much as a computer runs a program, and the brain is the machine doing it., cyberneticscybernetics, n.: The study of how animals, machines and organizations steer themselves using feedback: sensing the result of an action and correcting course. It was both a parent and an early rival of AI., David RumelhartDavid Rumelhart, prop. n.: An American psychologist who, with Geoffrey Hinton and Ronald Williams, showed in 1986 how to train multilayer neural networks, launching the modern revival of the field., deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems., Donald HebbDonald Hebb, prop. n.: A Canadian psychologist who proposed in 1949 that connections between brain cells strengthen when the cells are active together, an idea behind many learning rules for neural networks., Frank RosenblattFrank Rosenblatt, prop. n.: An American psychologist who invented the perceptron, the first neural network that learned from examples, and built a machine that learned to recognize simple images., good old-fashioned AIgood old-fashioned AI, n.: A nickname for the classic style of AI, dominant from the 1950s to the 1980s, that tried to make computers intelligent by writing knowledge and rules down by hand., Hebbian learningHebbian learning, n.: The idea that a connection between two brain cells grows stronger when they are active at the same time — the principle behind the slogan 'cells that fire together wire together'., Hubert DreyfusHubert Dreyfus, prop. n.: An American philosopher who argued from the 1960s that computers following explicit rules could never match human intelligence, which rests on bodily, intuitive know-how., John HopfieldJohn Hopfield, prop. n.: An American physicist who showed in 1982 how a network of simple units can store and recall memories, and who shared the 2024 Nobel Prize in Physics for work that enabled machine learning., McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic., neuro-symbolic AIneuro-symbolic AI, n.: AI that combines neural networks, which learn patterns from data, with symbolic methods, which follow explicit rules and logic, aiming for systems that both learn flexibly and reason reliably., Perceptrons (book)Perceptrons (book), prop. n.: A 1969 book by Marvin Minsky and Seymour Papert proving that simple one-layer neural networks cannot learn certain patterns. It is widely blamed for a decade-long slump in neural-network research., physical symbol system hypothesisphysical symbol system hypothesis, n.: The 1976 claim by two AI pioneers that a machine which stores and rearranges symbols according to rules has everything needed for general intelligence, and that anything generally intelligent must be such a machine., Warren McCullochWarren McCulloch, prop. n.: An American brain scientist who, with Walter Pitts, wrote the 1943 paper that first described nerve cells as simple logic units, the seed of today's artificial neural networks., binding problembinding problem, n.: The puzzle of how a brain, or a neural network, keeps track of which features belong together — that the red goes with the apple and the green with the leaf — when they are processed separately., connectomeconnectome, n.: A full chart of how a nervous system is wired: which nerve cells, or which brain regions, are linked to which others., Jerry FodorJerry Fodor, prop. n.: An American philosopher who argued that thinking is computation in an inner 'language of thought', and that neural networks cannot explain the structured nature of thought., language of thought hypothesislanguage of thought hypothesis, n.: The theory that we think in an inner, wordless mental language with its own symbols and grammar, which the brain manipulates when we reason, much as a computer manipulates code., Mark I PerceptronMark I Perceptron, prop. n.: A room-sized machine built around 1960 that learned to recognize simple shapes and letters through a camera made of 400 light sensors. It was one of the first machines built to learn from examples., neats and scruffiesneats and scruffies, phr.: A joking split of AI researchers into 'neats', who want clean mathematical theories that can be proved correct, and 'scruffies', who build messy collections of tricks that simply work.