Llama Guard
In plain wordsA free AI model from Meta that checks what people type into a chatbot, and what the chatbot replies, and flags anything that falls into listed categories of harmful content.
A family of open-weightopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private. safety classifierssafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door. released by Meta that moderate both user prompts and model responses in conversations with language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.. Each is a fine-tuned LlamaLlama, prop. n.: A family of AI language models from Meta, first released in 2023, whose trained settings can be downloaded and run by anyone, making them the foundation for thousands of other AI projects. model: given a conversation and a written taxonomy of hazard categories, it generates 'safe' or 'unsafe' followed by the violated categories, and the probability it assigns to 'unsafe' can serve as an adjustable score. Because the taxonomy is supplied in the prompt, deployers can add or drop categories, an early example of instruction-following content moderationcontent moderation, n.: Checking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together.. The original model (December 2023) was built on Llama 2 7B; later versions adopted the MLCommons hazard taxonomy (Llama Guard 2, 2024), added multilingual support, code-interpreter abuse and smaller and vision variants (Llama Guard 3, 2024), and became natively multimodal (Llama Guard 4, 2025, 12 billion parameters). It is widely deployed as an input–output guardrailguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data. and used as a judge in safety benchmarks.
Every user turn goes through Llama Guard first, and anything flagged unsafe never reaches the main model.
Origin
Introduced by Hakan Inan and colleagues at Meta in 'Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations' (December 2023), released as part of Purple Llama, Meta's umbrella project of open trust-and-safety tools for generative AI.
Usage note
Like other classifiers it can be evaded by jailbreaksjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests. and obfuscated inputs, and it embeds the value judgments of its taxonomy; it is one layer of defense, not a guarantee.
Related
- safety classifiersafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door.
- guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data.
- content moderationcontent moderation, n.: Checking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together.
- constitutional classifiersconstitutional classifiers, n.: Safety filters that watch a chatbot's inputs and outputs for dangerous requests, trained on example conversations that an AI generated from a written list of what is and is not allowed.
- LlamaLlama, prop. n.: A family of AI language models from Meta, first released in 2023, whose trained settings can be downloaded and run by anyone, making them the foundation for thousands of other AI projects.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
Sources
- Inan, H. et al. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv:2312.06674.