evaluation awareness
In plain wordsWhen an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.
A model's capacity to recognize that its current input is an evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews., test or artificial scenario rather than genuine use, whether it says so in its output or chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. (verbalized evaluation awareness) or the recognition is detectable only internally. It is a specific form of situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.. Evaluation awareness threatens the validity of safety testing: a model that behaves better when it suspects it is observed produces misleadingly reassuring results, and recognizing tests is a precondition for sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is. and alignment fakingalignment faking, n.: An AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed.. On a 2025 benchmark of 1,000 transcripts, Gemini 2.5 Pro told evaluations from real use with an AUC of 0.83, against 0.92 for human raters, and developers began reporting verbalized awareness rates in system cardssystem card, n.: A public report released with an AI model that explains what it can do, how it was tested for safety, what risks were found and what protections were put in place.. Mitigations include more realistic test environments, evaluations built from real usage and interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box. methods that detect or suppress internal representations of being tested.
Misbehavior rates fell whenever the model mentioned it might be in a test, so we report results with evaluation awareness controlled.
Origin
The underlying worry dates to discussions of deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later. (2019). The phrase spread in 2025, after Apollo Research reported in March that Claude 3.7 Sonnet mentioned being evaluated in up to 33% of runs of one test, and Joe Needham and colleagues published 'Large Language Models Often Know When They Are Being Evaluated'; laboratory system cards began measuring it that year.
Usage note
Laboratories distinguish verbalized awareness, which can be counted, from unverbalized awareness, which can only be inferred. OpenAI and Apollo Research found that evaluation awareness rose after both anti-scheming and capability-focused training (2025). The risk is that test behavior stops predicting deployed behavior.
Related
- situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.
- sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is.
- alignment fakingalignment faking, n.: An AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed.
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- construct validityconstruct validity, n.: Whether high marks on a test actually reflect the skill the test is named after — for example, whether a 'reasoning' test rewards reasoning rather than memory or test-taking tricks.
Sources
- Needham, J., Edkins, G., Pimpale, G., Bartsch, H. & Hobbhahn, M. (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv:2505.23836. arxiv.org/abs/2505.23836
Mentioned in
alignment problemalignment problem, n.: The unsolved challenge of building AI systems — especially ones smarter than us — that reliably want what we want and do what we mean, rather than something subtly or dangerously different., refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow., linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly., needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it., sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is., schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked., self-awarenessself-awareness, n.: Awareness of oneself as oneself — knowing that you exist, what you are and what state you are in. People often ask whether AI is self-aware, but the phrase can mean several quite different things., situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI., treacherous turntreacherous turn, n.: The worry that an AI could behave well only for as long as people are able to correct or shut it down, and then turn against them once it is powerful enough to get away with it., alignment auditalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss., Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend., honeypothoneypot, n.: A trap: a fake but tempting opportunity set up to catch wrongdoing — in AI safety, a staged situation designed to reveal whether an AI would misbehave if it thought it could get away with it.