Skip to content
Last Dictionary
Plane
Training & Post-trainingDeep Learning

elastic weight consolidation

nounAlso: EWC

In plain wordsA way to help a neural network learn a new task without forgetting an old one, by making the settings that mattered most for the old task harder to change, as if tied down with springs.

  1. A regularization method for continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch. that makes the weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. an earlier task relied on harder to move, in proportion to how much that task depended on them. After training on task A, it estimates each parameter's importance with the diagonal of the Fisher information matrix, a measure of how sharply the loss on A depends on that parameter; while training on task B it adds a quadratic penalty pulling each parameter toward its task-A value, weighted by that importance. Its authors pictured the penalty as springs anchoring each weight to its old value, stiffer where the weight matters more; it can also be derived as a Gaussian (Laplace) approximation to the posterior over weights after task A, in a Bayesian treatment of sequential learning. In the original experiments it curbed catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on. across sequences of MNIST-based tasks and of Atari games, and it became a standard baseline. Its limits are the diagonal approximation, penalties that accumulate over many tasks, and weaker results than replay in class-incremental settings.

    With EWC the network kept most of its task-A accuracy after learning task B; without it, accuracy collapsed.

L(θ)=LB(θ)+∑iλ2 Fi(θi−θA,i∗)2\mathcal{L}(\theta) = \mathcal{L}_B(\theta) + \sum_i \frac{\lambda}{2}\, F_i \big(\theta_i - \theta^{*}_{A,i}\big)^2

Origin

Introduced by James Kirkpatrick, Razvan Pascanu, Raia Hadsell and colleagues at DeepMind in a paper posted in December 2016 and published in PNAS in 2017. The name alludes to synaptic consolidation in neuroscience, in which some synapses become less plastic to protect memories.

Usage note

Related penalty-based methods include synaptic intelligence (2017) and memory-aware synapses (2018). In LLM fine-tuning a KL penaltyKL penalty, n.: A cost added during training that grows as an AI model drifts away from its starting version, so it improves at the goal without forgetting how to write normally or gaming its scorer. toward the starting model plays an analogous protective role in output space rather than weight space.

Related

See also
  • catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.
  • continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch.
  • stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.
  • regularizationregularization, n.: Any technique that stops a model from memorizing its training examples too closely — often by penalizing complexity — so that it works better on new data.
  • Bayesian inferenceBayesian inference, n.: Reasoning that treats unknowns as uncertain beliefs and updates them with evidence using Bayes' theorem. The result is not one answer but a range of answers, each with a probability.
  • KL penaltyKL penalty, n.: A cost added during training that grows as an AI model drifts away from its starting version, so it improves at the goal without forgetting how to write normally or gaming its scorer.
Contrast with
  • experience replayexperience replay, n.: Storing an AI agent's past experiences in a memory bank and repeatedly learning from random samples of them, rather than only from what just happened.

Sources

  1. Kirkpatrick, J. et al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13), 3521–3526. arXiv:1612.00796. arxiv.org/abs/1612.00796

Mentioned in

catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on., continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch., stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.