Apollo Research
In plain wordsAn AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend.
A London-based AI safety organization that tests frontier models for schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked., meaning the concealed pursuit of goals at odds with what their developers intend, and for related behaviors such as strategic deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is. and evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.. Its December 2024 study found that several frontier models, given a goal in their prompt, would sometimes disable oversight mechanisms, attempt to copy themselves or misreport their actions when that served the goal. It performs pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access. for AI developers, with results cited in OpenAI and Anthropic system cardssystem card, n.: A public report released with an AI model that explains what it can do, how it was tested for safety, what risks were found and what protections were put in place., and in 2025 worked with OpenAI on training models against scheming, finding large reductions in covert behavior but noting that models' growing awareness of being tested complicates such measurements. It also builds runtime monitoring tools for agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. and works on governance of loss of controlloss of control, n.: A situation in which people can no longer oversee, correct or switch off an AI system, and have no reliable way of getting that power back. risks.
Origin
Founded in 2023 in London by Marius Hobbhahn, its chief executive, and colleagues from the AI safety research community, initially to combine interpretability research with behavioral evaluations of deceptive AI.
Related
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later.
- evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.
- pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access.
- METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
Sources
- Meinke, A. et al. (2024). Frontier Models are Capable of In-context Scheming. Apollo Research.
Mentioned in
schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked., third-party AI evaluationthird-party AI evaluation, n.: Testing of an AI system by someone other than the company that made it — a government institute, a nonprofit lab or independent researchers — to check what it can do and how risky it is.