emergence
In plain wordsWhen a whole shows properties or abilities that none of its parts has and that are hard to predict from them — like a traffic jam arising from individual cars, or new skills appearing in larger AI models.
- 1
Philosophy and complex systems
The appearance, in a system of many interacting parts, of properties or patterns that the parts lack and that are not obvious from them: flocking from individual birds' simple rules, liquidity from molecules, traffic jams from drivers. Philosophers distinguish weak emergence, in which the higher-level property follows from the parts and their interactions but can in practice be found only by simulation or observation, from strong emergence, in which no amount of lower-level knowledge would let one derive it and which may have causal powers of its own. Most scientists accept weak emergence; strong emergence is controversial, and consciousness is sometimes proposed as its only real example.
- 2
Machine learning
The appearance in AI systems of capabilities that were not explicitly trained or programmed, such as in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round. arising from next-word prediction, or, in the narrower sense of emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured., capabilities that appear abruptly as models are scaled up rather than improving smoothly. Whether such jumps are real or artifacts of all-or-nothing metrics is disputed, as is the safety significance of capabilities that cannot be forecast from smaller models. Related uses include emergent communication among learning agents and emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice..
Calling the skill emergent means nobody trained it in directly, not that it is mysterious.
Origin
The philosopher George Henry Lewes introduced 'emergent' in this sense in Problems of Life and Mind (1875), building on John Stuart Mill's contrast (1843) between additive and non-additive causes; Samuel Alexander and C. D. Broad developed British emergentism in the 1920s. Philip Anderson's essay 'More Is Different' (Science, 1972) shaped its later use in physics and AI.
Usage note
In AI writing 'emergent' often means only 'not explicitly designed', which is weaker than either philosophical sense, so readers should ask which is meant. Claims that consciousness or understanding 'emerges' in large models are especially contested.
Related
- emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.
- grokkinggrokking, n.: When a model that has merely memorized its training examples suddenly, long afterward, figures out the underlying rule and starts getting new examples right.
- emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice.
- artificial lifeartificial life, n.: The study of life by building lifelike systems — in computer simulations, robots or chemistry — to learn how living things organize themselves, reproduce and evolve, and what other kinds of life could exist.
- consciousnessconsciousness, n.: Having experiences: there being something it feels like to be you, as when you see red or feel pain. Whether any AI is or could become conscious is an open and fiercely debated question.
Sources
- Lewes, G. H. (1875). Problems of Life and Mind, vol. 2. Trübner.
- Anderson, P. W. (1972). More Is Different. Science 177(4047), 393–396.
- Wei, J. et al. (2022). Emergent Abilities of Large Language Models. Transactions on Machine Learning Research.
Mentioned in
Douglas HofstadterDouglas Hofstadter, prop. n.: An American cognitive scientist whose prize-winning book 'Gödel, Escher, Bach' inspired a generation to study minds and AI, and who studies analogy as the heart of thinking., emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured., emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice., philosophy of artificial intelligencephilosophy of artificial intelligence, n.: The branch of philosophy that asks what AI is and could be: whether machines can think, understand or be conscious, what intelligence is, and what we owe to and should expect from intelligent machines., artificial lifeartificial life, n.: The study of life by building lifelike systems — in computer simulations, robots or chemistry — to learn how living things organize themselves, reproduce and evolve, and what other kinds of life could exist., John HollandJohn Holland, prop. n.: An American scientist who invented genetic algorithms, which solve problems by breeding and mutating populations of candidate answers, and helped found the science of complex systems.