quality diversity
In plain wordsA family of evolution-inspired search methods that aim to collect many different good solutions — for example, many distinct ways for a robot to walk — rather than a single best one.
A class of optimization algorithms whose goal is a large, diverse collection of high-performing solutions, each the best found for a different combination of user-defined behavioral characteristics, rather than a single optimum. MAP-Elites, the canonical example, divides a behavior space — for a walking robot, how often each leg touches the ground — into cells, keeps the fittest solution found in each cell, and generates new candidates by mutating elites drawn from this archive. Novelty search, a precursor, rewards behavioral novelty alone and showed that ignoring the objective can avoid deceptive dead ends. Quality-diversity archives provide repertoires of alternatives that support rapid adaptation, such as robots that recover from damage by switching gaits, and stepping stones for open-ended search. Since 2022 they have been combined with language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., for example to generate diverse candidate programs or diverse adversarial prompts for red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them..
Origin
Joel Lehman and Kenneth Stanley introduced novelty search in 2008. Jean-Baptiste Mouret and Jeff Clune introduced MAP-Elites in 2015, the year Antoine Cully, Clune, Danesh Tarapore and Mouret published damage-recovering robots in Nature. Justin Pugh, Lisa Soros, Stanley and colleagues used the name 'quality diversity' in a 2015 GECCO paper and set out the framework in 2016.
Related
- evolutionary computationevolutionary computation, n.: Problem-solving by simulated evolution, much as breeders improve crops: keep many candidate answers, let the better ones produce slightly altered offspring, drop the worse, and repeat for many rounds.
- neuroevolutionneuroevolution, n.: Training or designing neural networks by simulated evolution — breeding and mutating whole networks and keeping the best performers — instead of, or alongside, the usual gradient-based training.
- genetic algorithmgenetic algorithm, n.: A problem-solving method that copies natural evolution: many candidate solutions compete, the best are 'bred' by mixing and randomly tweaking them, and over many generations the solutions improve.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- AlphaEvolveAlphaEvolve, prop. n.: A Google DeepMind system that combines AI language models with an evolution-like search to invent and improve computer programs, finding new results in mathematics and computing.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
Sources
- Mouret, J.-B. & Clune, J. (2015). Illuminating Search Spaces by Mapping Elites. arXiv preprint.
- Cully, A., Clune, J., Tarapore, D. & Mouret, J.-B. (2015). Robots That Can Adapt Like Animals. Nature 521, 503–507.
- Pugh, J. K., Soros, L. B. & Stanley, K. O. (2016). Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI 3:40.
Mentioned in
evolutionary computationevolutionary computation, n.: Problem-solving by simulated evolution, much as breeders improve crops: keep many candidate answers, let the better ones produce slightly altered offspring, drop the worse, and repeat for many rounds., fitness functionfitness function, n.: In evolution-inspired computing, the scoring rule that says how good each candidate solution is, which decides which candidates survive and get to 'reproduce'., neuroevolutionneuroevolution, n.: Training or designing neural networks by simulated evolution — breeding and mutating whole networks and keeping the best performers — instead of, or alongside, the usual gradient-based training.