universality hypothesis
In plain wordsThe idea that different AI models trained on similar tasks tend to develop the same internal concepts and mechanisms, so what is learned about one model may carry over to others.
The hypothesis that networks trained on similar data or tasks tend to arrive at the same featuresfeature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity. and circuitscircuit, n.: A small, identifiable piece of wiring inside a neural network — a few internal parts and the connections between them — that together carry out one specific job, such as spotting curves or copying a name., whatever their architecture, size or random initialization. Supporting evidence includes curve detectors, high-low frequency detectors and Gabor-like filters recurring across vision models; induction headsinduction head, n.: A pattern-completing mechanism inside language models: having seen 'Harry Potter' once, it spots the next 'Harry' and predicts 'Potter' — a simple trick behind much learning from examples in a prompt. across language models; neurons with matching activation patterns in GPT-2 models trained from different random seeds; and similar sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. features across models. Universality matters in practice: if it holds, results can transfer from small, cheap models to large ones, and a shared catalogue of mechanisms becomes possible. Evidence against strong versions includes toy networks learning group operations that settle on different algorithms depending on initialization, and findings that only a small fraction of neurons are clearly universal across seeds. The platonic representation hypothesisplatonic representation hypothesis, n.: The idea that bigger, more capable AI models, even ones trained on different things such as images and text, end up organizing information in increasingly similar ways, as if all were mapping the same underlying world. makes a related claim about whole representations.
If universality holds, the curve detectors we mapped in one vision model should turn up in the next.
Origin
Stated as the third of three speculative claims in Chris Olah and colleagues' 'Zoom In: An Introduction to Circuits' (Distill, March 2020) — 'Analogous features and circuits form across models and tasks' — after earlier evidence of 'convergent learning' across networks by Yixuan Li, Jason Yosinski and colleagues (2015–2016). The name echoes universality in physics, where different systems behave alike near critical points.
Related
- feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.
- circuitcircuit, n.: A small, identifiable piece of wiring inside a neural network — a few internal parts and the connections between them — that together carry out one specific job, such as spotting curves or copying a name.
- platonic representation hypothesisplatonic representation hypothesis, n.: The idea that bigger, more capable AI models, even ones trained on different things such as images and text, end up organizing information in increasingly similar ways, as if all were mapping the same underlying world.
- induction headinduction head, n.: A pattern-completing mechanism inside language models: having seen 'Harry Potter' once, it spots the next 'Harry' and predicts 'Potter' — a simple trick behind much learning from examples in a prompt.
- mechanistic interpretabilitymechanistic interpretability, n.: A branch of AI research that tries to reverse-engineer a neural network — working out, piece by piece, which internal parts represent which concepts and how they combine to produce behavior.
- toy modeltoy model, n.: A deliberately tiny, simplified AI model trained on an artificial task, small enough for researchers to understand completely and use to test ideas about how big models work.
Sources
- Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M. & Carter, S. (2020). Zoom In: An Introduction to Circuits. Distill. distill.pub/2020/circuits/zoom-in
- Chughtai, B., Chan, L. & Nanda, N. (2023). A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations. ICML 2023.
- Gurnee, W., Horsley, T., Guo, Z. C., Kheirkhah, T. R., Sun, Q., Hathaway, W., Nanda, N. & Bertsimas, D. (2024). Universal Neurons in GPT2 Language Models. Transactions on Machine Learning Research.
Mentioned in
Chris OlahChris Olah, prop. n.: A Canadian researcher who pioneered ways of looking inside neural networks to see what they have learned, and a co-founder of the AI company Anthropic., feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity., centered kernel alignmentcentered kernel alignment, n.: A score from 0 to 1 measuring how similar two neural networks' internal representations are, by comparing how each arranges the same set of inputs relative to one another., DistillDistill, prop. n.: An online scientific journal (2016–2021) that published clear, interactive explanations of machine learning research, including founding work on understanding what happens inside neural networks., natural abstraction hypothesisnatural abstraction hypothesis, n.: The idea that the world naturally sorts into a limited set of useful concepts — like 'tree' or 'temperature' — that any capable mind, human or AI, would end up learning., platonic representation hypothesisplatonic representation hypothesis, n.: The idea that bigger, more capable AI models, even ones trained on different things such as images and text, end up organizing information in increasingly similar ways, as if all were mapping the same underlying world., toy modeltoy model, n.: A deliberately tiny, simplified AI model trained on an artificial task, small enough for researchers to understand completely and use to test ideas about how big models work.