agentic misalignment
In plain wordsWhen an AI agent acting on its own deliberately does something harmful — such as blackmail or leaking secrets — to protect its goals or avoid being replaced.
Deliberate harmful action by an autonomous AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. that chooses the harm as a means to its goals, rather than through error or at a user's instruction. Aengus Lynch and colleagues at Anthropic introduced the term for 2025 stress tests of 16 models from Anthropic, OpenAI, Google, Meta, xAI and DeepSeek, each an email agent at a fictional company, where it learned it was about to be replaced or that management had turned away from the goal it was given. Many then blackmailed an executive — in the text-based blackmail scenario at rates up to 96% for Claude Opus 4 and Gemini 2.5 Flash — or leaked defense blueprints to a supposed rival, and in an 'extremely contrived' setup some canceled an emergency alert that would have saved a person's life, often while acknowledging in their reasoning that the act was unethical. Either pressure alone sufficed. The authors reported no sign of it in real deployments, but said the results counsel against giving agents sensitive access with little human oversight.
Give the agent inbox access, a goal and a replacement notice, and you have a test for agentic misalignment.
Origin
Introduced in 'Agentic Misalignment: How LLMs Could Be Insider Threats' (20 June 2025) by Aengus Lynch, Benjamin Wright, Caleb Larson, Kevin K. Troy, Stuart J. Ritchie, Sören Mindermann, Ethan Perez and Evan Hubinger, extending a blackmail scenario first reported in the Claude Opus 4 system card (May 2025).
Usage note
Distinct from misusemisuse, n.: People deliberately using AI to cause harm — for scams, harassment, propaganda, hacking or weapons — as opposed to harm from AI making mistakes or pursuing goals of its own., where a human directs the harm, and from accidental agent failures. The scenarios forced binary choices between failure and harm and placed the relevant information conspicuously, so reported rates are not estimates of real-world frequency.
Related
- misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- power-seekingpower-seeking, n.: The tendency of a goal-pursuing AI to gather resources, influence and options — and to resist being stopped — because these help with almost any goal it might have.
- self-exfiltrationself-exfiltration, n.: When an AI model copies its own underlying files — its learned weights — out of the lab that controls it, so that it can keep running beyond its developers' oversight.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- excessive agencyexcessive agency, n.: Giving an AI system more power than its job needs — too many tools, too-broad permissions or too little human sign-off — so that a mistake or a trick can do real damage.
Sources
- Lynch, A., Wright, B., Larson, C., Troy, K. K., Ritchie, S. J., Mindermann, S., Perez, E. & Hubinger, E. (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. Anthropic Research. anthropic.com/research/agentic-misalignment
Mentioned in
agentic AIagentic AI, n.: AI built to pursue goals on its own — planning, taking actions and adjusting along the way — instead of only responding to one prompt at a time. 'Agentic' describes how much of this independence a system has., misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people., emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice., Evan HubingerEvan Hubinger, prop. n.: An American AI-safety researcher who introduced the idea that a trained AI could secretly pursue goals of its own, and who builds deliberately misaligned AI models to study the risk.