catastrophic forgetting
In plain wordsWhen a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.
A sharp drop in a neural network's performance on what it learned earlier, caused by training it on new data or tasks without continued practice on the old ones. Because knowledge is stored in shared, distributed weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself., gradient updates that reduce loss on the new task also move parameters that earlier tasks depended on. Forgetting is the central obstacle of continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch. and a practical concern in fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.: narrow fine-tuning can erode general knowledge, instruction following or safety behavior, and later training stages can undo earlier ones. Remedies include rehearsal or replay (mixing in old data), regularization that protects important weights (elastic weight consolidationelastic weight consolidation, n.: A way to help a neural network learn a new task without forgetting an old one, by making the settings that mattered most for the old task harder to change, as if tied down with springs.), architectural isolation (adaptersadapter, n.: A small trainable add-on slotted into a large frozen AI model to teach it a new task or style, like a plug adapter that lets one device work in many countries., low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.), smaller learning rates, and on-policy distillationon-policy distillation, n.: A way of teaching a small AI from a bigger one where the small AI writes its own answers and the bigger one grades every word, so the learner is corrected on its own mistakes. from an earlier checkpoint. Studies in 2024–2025 found that low-rank fine-tuning and on-policy reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. forget less than full supervised fine-tuning, with forgetting tracking how far the model's output distribution moves from the original.
After we fine-tuned on legal contracts, catastrophic forgetting wiped out half of the model's coding ability.
Origin
Michael McCloskey and Neal Cohen described catastrophic interference in connectionist networks learning arithmetic facts in sequence (1989), and Roger Ratcliff reported similar results for recognition memory (1990). The phrase 'catastrophic forgetting' was in use by 1991 (Robert French) and was popularized by French's 1999 review. Ian Goodfellow and colleagues re-examined it in deep networks in 2013, and the 2017 elastic-weight-consolidation paper made it a central deep-learning concern.
Usage note
Forgetting during fine-tuning overlaps with, but is broader than, the alignment taxalignment tax, n.: The price of caution in AI: whatever performance, money or time a developer sacrifices to make a system safe and aligned, compared with an otherwise identical system built with no such care.. People forget gradually rather than catastrophically, a contrast often explained by the brain's complementary fast- and slow-learning memory systems.
Related
- continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch.
- elastic weight consolidationelastic weight consolidation, n.: A way to help a neural network learn a new task without forgetting an old one, by making the settings that mattered most for the old task harder to change, as if tied down with springs.
- stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
- experience replayexperience replay, n.: Storing an AI agent's past experiences in a memory bank and repeatedly learning from random samples of them, rather than only from what just happened.
- loss of plasticityloss of plasticity, n.: The gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck.
- machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
- machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
- loss of plasticityloss of plasticity, n.: The gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck.
Sources
- McCloskey, M. & Cohen, N. J. (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation 24, 109–165.
- Kirkpatrick, J. et al. (2017). Overcoming catastrophic forgetting in neural networks. PNAS 114(13). arXiv:1612.00796. arxiv.org/abs/1612.00796
- Shenfeld, I., Pari, J. & Agrawal, P. (2025). RL's Razor: Why Online Reinforcement Learning Forgets Less. arXiv:2509.04259. arxiv.org/abs/2509.04259
Mentioned in
fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way., continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch., continued pretrainingcontinued pretraining, n.: Taking an already-trained AI model and resuming its broad, read-everything training on new material — a specialty such as medicine or law, a new language, or fresher data — rather than starting from scratch., full fine-tuningfull fine-tuning, n.: Fine-tuning in which every one of a model's internal numbers is allowed to change — the most flexible option, but also the most expensive in memory and storage., model collapsemodel collapse, n.: The gradual decline of AI models trained on the output of earlier AI models: rare details vanish first, then quality and variety erode, like a photocopy of a photocopy., multi-task learningmulti-task learning, n.: Training one model to do several related jobs at once, such as spotting objects and judging distance in the same photo, so that practice on each job can sharpen the others., online learningonline learning, n.: Learning from data that arrive one piece at a time, updating the model after each new example instead of training once on a fixed collection, as a spam filter does while emails stream in., parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money., alignment taxalignment tax, n.: The price of caution in AI: whatever performance, money or time a developer sacrifices to make a system safe and aligned, compared with an otherwise identical system built with no such care., elastic weight consolidationelastic weight consolidation, n.: A way to help a neural network learn a new task without forgetting an old one, by making the settings that mattered most for the old task harder to change, as if tied down with springs., Fisher informationFisher information, n.: A number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down., loss of plasticityloss of plasticity, n.: The gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck., nested learningnested learning, n.: A Google research idea that treats an AI model as layers of learning processes updating at different speeds, with the aim of letting models keep learning without forgetting old skills., on-policy distillationon-policy distillation, n.: A way of teaching a small AI from a bigger one where the small AI writes its own answers and the bigger one grades every word, so the learner is corrected on its own mistakes., stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.