recursive self-improvement
In plain wordsAn AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion.
Improvement of an AI system carried out by the system itself, by rewriting its code, redesigning its architecture, generating its own training data or doing the research that produces its successor, in a loop where each gain makes the next one easier. Early writing pictured a single program editing itself; current discussion more often concerns a looser loop in which AI agents automate growing parts of the research and engineering pipeline at a frontier labfrontier lab, n.: One of the handful of companies building the most advanced AI models of the moment, spending billions of dollars on computing power to train them., with humans still supervising. Partial forms are already routine: models generate synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars., provide feedback for trainingreinforcement learning from AI feedback, n.: Training an AI with preference judgments made by another AI instead of by people, which is cheaper and faster but only as good as the AI doing the judging. and write much of the code used to train successors, and evolutionary coding agents such as Google DeepMind's AlphaEvolveAlphaEvolve, prop. n.: A Google DeepMind system that combines AI language models with an evolution-like search to invent and improve computer programs, finding new results in mathematics and computing. (2025) have found speedups in the kernels used to train their own model family. Whether such loops compound or saturate is the central question for takeofftakeoff, n.: The stretch of time between AI reaching roughly human level and AI becoming vastly superhuman. People argue over whether it would take decades, years, months or just days. speed. Several frontier safety frameworksresponsible scaling policy, n.: A public rulebook in which an AI company promises to test each new model for dangerous abilities and to add stronger safeguards, or slow down, as its models become more capable. treat substantial automation of AI R&D as a capability thresholdcapability threshold, n.: A level of dangerous ability — such as meaningfully helping someone make a biological weapon — that, once an AI model reaches it, obliges its developer to add stronger safeguards or hold the model back. requiring extra safeguards.
The worry is not one model rewriting its own weights but a lab where agents do most of the research on their successors.
Origin
The idea is implicit in I. J. Good's 1965 ultraintelligent machine and in Jürgen Schmidhuber's 1987 work on self-referential learning. Eliezer Yudkowsky's writing on seed AIseed AI, n.: An early AI built mainly to understand and upgrade itself, so that it could grow, like a seed, into a far more capable mind. from 2001 spread it (his Creating Friendly AI spoke of 'recursive self-enhancement'); the exact phrase, whose earliest use is not established, was standard in AI-risk writing by 2008, the year of Yudkowsky's essay 'Recursive Self-Improvement'. It entered industry roadmaps from 2024 as labs set goals for automating AI research.
Related
- intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.
- automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors.
- seed AIseed AI, n.: An early AI built mainly to understand and upgrade itself, so that it could grow, like a seed, into a far more capable mind.
- Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement.
- takeofftakeoff, n.: The stretch of time between AI reaching roughly human level and AI becoming vastly superhuman. People argue over whether it would take decades, years, months or just days.
- capability thresholdcapability threshold, n.: A level of dangerous ability — such as meaningfully helping someone make a biological weapon — that, once an AI model reaches it, obliges its developer to add stronger safeguards or hold the model back.
Mentioned in
AI scientistAI scientist, n.: An AI system meant to do research largely on its own: reading the literature, coming up with ideas, running experiments and writing up the results., automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors., fast takeofffast takeoff, n.: A scenario in which AI goes from about human level to far beyond it very quickly, in days, weeks or a few months, leaving people little time to react., I. J. GoodI. J. Good, prop. n.: A British statistician who broke codes with Alan Turing and in 1965 argued that a machine smarter than us could design still smarter machines, setting off an 'intelligence explosion'., intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time., self-improvementself-improvement, n.: When an AI gets better by learning from material it produces itself — its own answers, critiques or practice problems — rather than only from new human-made examples., Darwin Gödel MachineDarwin Gödel Machine, prop. n.: An experimental AI coding assistant that improves itself by rewriting its own code, testing each new version and keeping a family tree of variants to build on., FOOMFOOM, n.: Slang for a sudden, explosive jump in an AI's abilities, the AI going 'foom' and racing from about human level to vastly superhuman in a very short time., friendly AIfriendly AI, n.: An early name for AI built so that it reliably benefits people rather than harming them. The problem it named is now usually called AI alignment., Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement., RE-BenchRE-Bench, prop. n.: A test from the research group METR that pits AI agents against human experts on seven open-ended machine-learning engineering projects, comparing what each achieves in the same time., research tasteresearch taste, n.: The knack good researchers have for choosing worthwhile problems, designing the right experiments and drawing the right conclusions — a skill seen as key to whether AI can do research on its own., seed AIseed AI, n.: An early AI built mainly to understand and upgrade itself, so that it could grow, like a seed, into a far more capable mind., sharp left turnsharp left turn, n.: A feared moment when an AI's abilities suddenly leap far beyond its training while its good behavior fails to keep up, leaving it powerful but no longer aligned.