Skip to content
Last Dictionary
Plane
Machine LearningDeep LearningReinforcement Learning

model-agnostic meta-learning

nounAlso: MAML, first-order MAML, FOMAML, Reptile

In plain wordsA training method that searches for a starting point for a model's internal settings from which it can learn any new task of a given kind with just a few small adjustments.

  1. An optimization-based meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples. algorithm that learns a starting point rather than a finished model: shared initial parameters from which a handful of gradient descentgradient descent, n.: A method for training models by repeatedly nudging every setting a small step in whichever direction most reduces the error, like a hiker who cannot see the valley but keeps stepping wherever the ground tilts down. updates on a few examples fit any new task of the training family well. Training proceeds over episodes sampled from a distribution of tasks. For each task an inner loop takes one or more gradient steps from the shared initialization on the task's support examples; an outer loop then updates the initialization to reduce the loss on the task's query examples after adaptation, which requires differentiating through the inner updates (second-order derivatives). The name signals that it works with any architecture trained by gradient descent, whether a reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. policy, a regressor or a classifier. First-order approximations (FOMAML) and Reptile, which simply moves the initialization toward task-adapted weights, avoid the second-order cost. MAML became a standard method for few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.; later analyses found that much of its benefit comes from learning reusable features rather than rapid adaptation, and that simple pretrained baselines often match it.

    With MAML the grasping policy adapted to a new object after three gradient steps on ten demonstrations.

θ←θ−β ∇θ∑TiLTi ⁣(θ−α∇θLTi(θ))\theta \leftarrow \theta - \beta \, \nabla_{\theta} \sum_{\mathcal{T}_i} \mathcal{L}_{\mathcal{T}_i}\!\big(\theta - \alpha \nabla_{\theta} \mathcal{L}_{\mathcal{T}_i}(\theta)\big)

Origin

Introduced by Chelsea Finn, Pieter Abbeel and Sergey Levine in 'Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks' (ICML 2017). Alex Nichol, Joshua Achiam and John Schulman proposed the first-order Reptile algorithm in 2018, and Aniruddh Raghu and colleagues' 'Rapid Learning or Feature Reuse?' (2020) analyzed what MAML actually learns.

Related

See also
  • meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples.
  • few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.
  • gradient descentgradient descent, n.: A method for training models by repeatedly nudging every setting a small step in whichever direction most reduces the error, like a hiker who cannot see the valley but keeps stepping wherever the ground tilts down.
  • transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data.
  • fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
  • multi-task learningmulti-task learning, n.: Training one model to do several related jobs at once, such as spotting objects and judging distance in the same photo, so that practice on each job can sharpen the others.

Sources

  1. Finn, C., Abbeel, P. & Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Proc. ICML 2017.
  2. Nichol, A., Achiam, J. & Schulman, J. (2018). On First-Order Meta-Learning Algorithms. arXiv preprint.
  3. Raghu, A., Raghu, M., Bengio, S. & Vinyals, O. (2020). Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML. Proc. ICLR 2020.

Mentioned in

few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice., meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples., Chelsea FinnChelsea Finn, prop. n.: An American computer scientist at Stanford known for 'learning to learn' methods that let AI adapt to new tasks from a few examples, and a co-founder of the robotics company Physical Intelligence., Pieter AbbeelPieter Abbeel, prop. n.: A Belgian-American roboticist at Berkeley whose research taught robots to learn from demonstrations and from trial and error, and who co-founded the robotics company Covariant., Sergey LevineSergey Levine, prop. n.: A Berkeley computer scientist, one of the most prolific researchers on how robots can learn skills from data and experience, and a co-founder of the robotics company Physical Intelligence.