helpful, honest and harmless
In plain wordsA three-part goal for AI assistants, popularized by Anthropic: be genuinely useful, tell the truth, and avoid causing harm — shorthand for what an aligned chatbot should be.
A set of three criteria for aligned AI assistants, used as design goals, training targets and evaluation categories. As Amanda Askell and colleagues spelled them out in 2021, helpfulness is a real, efficient attempt at what the user asked, with follow-up questions when information is missing; honesty is accurate information whose uncertainty is calibrated and stated without misleading; and harmlessness is avoiding offensive or discriminatory output and declining to aid dangerous acts. The criteria can conflict — maximal help to one user may harm others, and refusing is harmless but unhelpful — so much post-training work, from reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well. on separate helpfulness and harmlessness datasets to Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand., concerns trading them off. The framing shaped early assistant models and later model specificationsmodel specification, n.: A public document in which an AI company spells out how its models are supposed to behave — what they should do, refuse and prioritize, and whom to obey when instructions conflict..
Our preference data scores each response separately on helpful, honest and harmless.
Origin
Introduced by Amanda Askell, Yuntao Bai, Anna Chen and colleagues at Anthropic in 'A General Language Assistant as a Laboratory for Alignment' (December 2021), and used to structure Anthropic's 2022 papers on RLHF and Constitutional AI.
Usage note
Critics argue the triad is vague and that harmlessness was often operationalized as excessive refusal (over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program.); later frameworks stress helpfulness within explicit limits, as in safe completionssafe completion, n.: A way of training AI to handle sensitive requests by giving the most helpful answer that stays safe, instead of simply choosing between answering fully and refusing..
Related
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
- AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this.
- Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand.
- reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.
- refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
- over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program.
- model specificationmodel specification, n.: A public document in which an AI company spells out how its models are supposed to behave — what they should do, refuse and prioritize, and whom to obey when instructions conflict.
Sources
- Askell, A., Bai, Y., Chen, A. et al. (2021). A General Language Assistant as a Laboratory for Alignment. arXiv:2112.00861. arxiv.org/abs/2112.00861
- Bai, Y. et al. (2022). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv:2204.05862. arxiv.org/abs/2204.05862
Mentioned in
AI assistantAI assistant, n.: An AI program, such as ChatGPT, Claude or Gemini, that helps a person through conversation: answering questions, writing, summarizing, coding and, increasingly, taking actions on their behalf., Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade., over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program., context distillationcontext distillation, n.: Training a model to behave as if it had been given a long set of instructions, so it no longer needs to read those instructions every time., helpful-only modelhelpful-only model, n.: A version of an AI trained to be helpful but without the safety training that makes it refuse harmful requests, kept in-house so testers can see what the model could really do.