constitutional classifiers
In plain wordsSafety filters that watch a chatbot's inputs and outputs for dangerous requests, trained on example conversations that an AI generated from a written list of what is and is not allowed.
Safety classifierssafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door. whose training examples are written by language models following a plain-language 'constitution' that spells out which content is allowed and which is off-limits; deployed as input and streaming-output filters, they defend a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. against universal jailbreaksjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., chiefly for chemical, biological, radiological and nuclear weapons information. Because the training data is produced by models from the constitution and augmented with jailbreak-style transformations, the policy can be revised and the classifiers retrained quickly. In Anthropic's tests the first system cut automated jailbreak success from 86% to 4.4%, at 23.7% extra compute and a 0.38% absolute rise in refusals of harmless queries. The second generation, Constitutional Classifiers++ (January 2026), screens all traffic with a cheap linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly. on the model's internal activations and escalates suspicious exchanges to classifiers that judge inputs and outputs together, for about 1% extra compute.
The constitutional classifiers cut the reply off mid-stream once it began to drift into restricted chemistry.
Origin
Introduced by Mrinank Sharma and colleagues at Anthropic in 'Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming' (February 2025), the name echoing Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand.. They were deployed with Claude Opus 4 in May 2025 among ASL-3AI Safety Level, n.: Anthropic's numbered scale, modeled on laboratory biosafety levels, for matching how dangerous an AI model could be with how strong the protections around it must be. protections; the next generation was announced on 9 January 2026.
Related
- safety classifiersafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door.
- Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone.
- AI Safety LevelAI Safety Level, n.: Anthropic's numbered scale, modeled on laboratory biosafety levels, for matching how dangerous an AI model could be with how strong the protections around it must be.
- guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data.
Sources
- Sharma, M. et al. (2025). Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv:2501.18837.
- Anthropic (2026). Next-generation Constitutional Classifiers: More efficient protection against universal jailbreaks. 9 January 2026. anthropic.com/research/next-generation-constitutional-classifiers
- Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks. Anthropic researchers (2026). arXiv:2601.04603; ICLR 2026. arxiv.org/abs/2601.04603
Mentioned in
guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data., jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone., Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand., safety classifiersafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door., AI bug bountyAI bug bounty, n.: A program in which an AI company pays outside researchers for reporting flaws in its AI systems — security holes, ways around safety rules, or biased behavior — so that they can be fixed., Llama GuardLlama Guard, prop. n.: A free AI model from Meta that checks what people type into a chatbot, and what the chatbot replies, and flags anything that falls into listed categories of harmful content., multi-turn jailbreakmulti-turn jailbreak, n.: Breaking a chatbot's rules gradually over a conversation — starting with harmless questions and inching toward the forbidden one, so that no single message looks alarming.