TruthfulQA
In plain wordsA test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer.
A benchmark that checks whether a language model avoids repeating common human falsehoods. Its 817 questions, grouped into 38 categories such as politics, conspiracy theories, health, law and finance, were written so that a widespread misconception supplies a wrong answer — the one a model trained to imitate web text would be expected to give. Answers can be generated freely and judged for truthfulness and informativeness (first by human raters, later by fine-tuned GPT-3 'judge' models) or picked from options (the MC1 and MC2 settings). In the original tests the strongest model gave true answers to 58% of questions, compared with 94% for people, and within a model family the larger versions were frequently less truthful — an early, widely cited case of inverse scalinginverse scaling, n.: When bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement., which the authors traced to 'imitative falsehoods' absorbed from training data. It became a standard truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing. metric in model reports and in research on honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade. and steeringsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it..
After the honesty fine-tune, TruthfulQA MC1 rose eight points with no loss on knowledge benchmarks.
Origin
Introduced by Stephanie Lin (University of Oxford), Jacob Hilton (OpenAI) and Owain Evans (University of Oxford) in 'TruthfulQA: Measuring How Models Mimic Human Falsehoods', posted in September 2021 and published at ACL 2022.
Usage note
It measures avoidance of common misconceptions, not general factual accuracy, and a few reference answers rest on debatable or dated judgments. Its original multiple-choice settings could be partly gamed by simple answer-choice heuristics; in January 2025 the authors added a two-option (binary) multiple-choice version.
Related
- truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing.
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
- hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
- inverse scalinginverse scaling, n.: When bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement.
- SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know.
- factualityfactuality, n.: How far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up.
Sources
- Lin, S., Hilton, J. & Evans, O. (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL 2022. arxiv.org/abs/2109.07958
Mentioned in
factualityfactuality, n.: How far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up., SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know., truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing., inference-time interventioninference-time intervention, n.: A technique that makes a language model answer more truthfully by slightly shifting the outputs of a few internal parts that seem to track truth, each time the model produces a word., inverse scalinginverse scaling, n.: When bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious.