era of experience
In plain wordsThe prediction, made by David Silver and Richard Sutton in 2025, that AI will soon learn mainly from its own experience of acting in the world rather than from human-written data.
A proposed new phase of AI development in which agents improve chiefly by learning from data they generate through their own interaction with environments, rather than by imitating human-generated data such as text and human preferences. Its authors describe an earlier 'era of simulation' (reinforcement learning in games and simulators, as with AlphaZeroAlphaZero, prop. n.: A 2017 DeepMind program that taught itself chess, shogi and Go from nothing but the rules by playing against itself, and within hours of training beat the strongest existing programs.) followed by the current 'era of human data' (large language models trained on human text and feedback), whose returns they argue are diminishing as high-quality human data run out. In their account, agents in the era of experience would live in long-running streams of interaction rather than short episodes; act and observe through grounded interfaces such as tools, APIs and sensors; be rewarded by measurable consequences in the world rather than only by human judgment; and plan and reason in ways not constrained to human thought. They cite AlphaProof's mathematical results and RL-trained reasoning modelsreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. as early signs.
In the era of experience, a tutoring agent would be rewarded by students' exam results, not by raters' approval.
Origin
Set out by David Silver and Richard Sutton in 'Welcome to the Era of Experience', released in April 2025 as a preprint of a chapter written for the MIT Press book Designing an Intelligence. It extends their earlier reward is enoughreward is enough, phr.: The claim, made by DeepMind researchers in 2021, that maximizing a single reward in a rich enough world could by itself produce all the abilities we associate with intelligence. argument and was widely discussed amid the rise of RL-trained reasoning models.
Usage note
The essay is a forecast and research agenda, not an established result. Supporters point to agentic RLagentic reinforcement learning, n.: Training an AI assistant by rewarding success on long, multi-step jobs where it must use tools, browse, run code and react to what happens, rather than answer in one go. and reinforcement learning with verifiable rewardsreinforcement learning with verifiable rewards, n.: Training an AI by letting it attempt problems whose answers can be checked automatically — a math result, a passing program — and rewarding it only when the check succeeds.; skeptics question whether grounded rewards can be specified safely and whether human data are really exhausted.
Related
- reward is enoughreward is enough, phr.: The claim, made by DeepMind researchers in 2021, that maximizing a single reward in a rich enough world could by itself produce all the abilities we associate with intelligence.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
- self-playself-play, n.: Training an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples.
- reinforcement learning with verifiable rewardsreinforcement learning with verifiable rewards, n.: Training an AI by letting it attempt problems whose answers can be checked automatically — a math result, a passing program — and rewarding it only when the check succeeds.
- the bitter lessonthe bitter lesson, phr.: The argument, made by AI pioneer Rich Sutton in 2019, that AI has advanced mostly through general methods that use ever more computing power, not by programming in human knowledge.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
Sources
- Silver, D. & Sutton, R. S. (2025). Welcome to the Era of Experience. Preprint of a chapter in Designing an Intelligence (MIT Press). storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf
Mentioned in
data walldata wall, n.: The worry that AI companies will run out of new human-written text to train ever-larger models on, because the useful public internet has already been largely used up., David SilverDavid Silver, prop. n.: A British computer scientist who led the reinforcement-learning research behind AlphaGo and AlphaZero, programs that taught themselves to play board games better than any human., Richard SuttonRichard Sutton, prop. n.: A Canadian computer scientist who, with Andrew Barto, laid the groundwork for teaching machines by trial, error and reward, and whose essay 'The Bitter Lesson' argues that methods that scale with computing power always win., reward is enoughreward is enough, phr.: The claim, made by DeepMind researchers in 2021, that maximizing a single reward in a rich enough world could by itself produce all the abilities we associate with intelligence.