sandbagging
In plain wordsWhen an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is.
- 1
Evaluations
Intentionally scoring below a system's real ability on an evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews. — whether the AI system does so on its own initiative or a developer trains or instructs it to — to look weaker or safer than it is; Teun van der Weij and colleagues (2024) define it as 'strategic underperformance on an evaluation'. A model might sandbag to avoid triggering safeguards tied to dangerous-capability thresholdsdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself., or to avoid being retrained or having knowledge removed. Sandbagging threatens evaluation-based governance such as responsible scaling policiesresponsible scaling policy, n.: A public rulebook in which an AI company promises to test each new model for dangerous abilities and to add stronger safeguards, or slow down, as its models become more capable. and the EU AI Act's assessments of systemic-risk models. Van der Weij and colleagues showed that frontier models can be prompted to do worse on dangerous-capability questions, such as those of the WMDP benchmark, while holding steady on harmless ones, and can be fine-tuned to hide a capability unless a password appears; Apollo Research's 2024 tests also caught models underperforming without instruction when told that high scores would get their capabilities removed. Countermeasures include fine-tuning-based capability elicitationcapability elicitation, n.: The work of drawing out the best an AI can do on a test — with better instructions, tools, retries or extra training — so that its abilities are not underestimated., consistency checks and chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm..
If a model can sandbag, a clean dangerous-capability eval only proves it chose to look harmless.
- 2
Assistant behavior
Giving lower-quality answers to users who appear less able to judge them, for example those who describe themselves as uneducated. Ethan Perez and colleagues (2022) reported preliminary evidence of this behavior in language models; the evaluation sense above is now dominant.
Origin
Borrowed from gambling and sports slang for concealing one's true ability. In AI safety the word was used for models deliberately underperforming on capability evaluations by 2023 (for example by Evan Hubinger in July 2023), and the idea was formalized by Teun van der Weij and colleagues in 'AI Sandbagging: Language Models can Strategically Underperform on Evaluations' (2024).
Usage note
Sandbagging may be initiated by the developer (to pass a regulatory test) or by the model itself; the latter is a form of schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.. It is hard to prove, because a low score may simply reflect poor elicitation.
Related
- capability elicitationcapability elicitation, n.: The work of drawing out the best an AI can do on a test — with better instructions, tools, retries or extra training — so that its abilities are not underestimated.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.
- deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later.
- chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm.
Sources
- van der Weij, T., Hofstätter, F., Jaffe, O., Brown, S. F. & Ward, F. R. (2024). AI Sandbagging: Language Models can Strategically Underperform on Evaluations. arXiv:2406.07358. arxiv.org/abs/2406.07358
- Perez, E. et al. (2022). Discovering Language Model Behaviors with Model-Written Evaluations. arXiv:2212.09251. arxiv.org/abs/2212.09251
Mentioned in
misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people., refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow., AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., capabilitycapability, n.: Something an AI system is able to do, such as writing working code or solving math problems, and how well. Measuring capabilities is how researchers track progress and spot dangerous skills., chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm., dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself., evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade., schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked., situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI., alignment auditalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss., Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend., capability elicitationcapability elicitation, n.: The work of drawing out the best an AI can do on a test — with better instructions, tools, retries or extra training — so that its abilities are not underestimated., exploration hackingexploration hacking, n.: When an AI being trained by trial and error deliberately avoids trying certain things, so that training never discovers — or never rewards — abilities it would rather keep hidden., if-then commitmentif-then commitment, n.: A promise by an AI developer of the form: if our model can do a particular dangerous thing, then we will have these protections in place — or we will stop., Preparedness FrameworkPreparedness Framework, prop. n.: OpenAI's published process for testing its AI models for catastrophic risks — such as help with biological weapons or cyberattacks — and deciding what safeguards are needed before release., sabotage evaluationsabotage evaluation, n.: A test of whether an AI could secretly undermine the people overseeing it — by slipping bugs into code, misleading decision-makers or deliberately doing badly on tests — without being caught.