factuality
In plain wordsHow far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up.
The property that generated statements are true with respect to established world knowledge, and the family of methods for measuring it. Researchers distinguish factuality from faithfulness — consistency with a given source document — since a summary can be faithful to an erroneous source, or factual while adding unsupported content. Short-form factuality is measured with question sets that have single verifiable answers, such as SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know. and TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer.; long-form factuality decomposes generated text into atomic claims and checks each against a knowledge source, as in FActScore (2023) and search-augmented evaluators. Factuality depends on parametric knowledgeparametric knowledge, n.: What an AI model 'knows' from training, stored in the numbers of its network rather than looked up in documents — like facts you have memorized rather than ones you check in a book., calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time. and willingness to abstain, and is improved by retrieval and groundinggrounding, n.: Connecting an AI's words to something real — trusted documents, a live search, images or the physical world — so that its answers rest on evidence rather than only on what it absorbed in training., training that rewards abstentionabstention, n.: When an AI declines to answer because it is not sure enough — saying 'I don't know' or asking for more information — instead of guessing and risking a confident mistake. over guessing, and post-hoc verification.
Factuality on biographies dropped sharply for people with little web presence.
Origin
Used in summarization research from about 2019–2020, notably in Joshua Maynez and colleagues' study of faithfulness and factuality in abstractive summarization (2020), and extended to open-ended LLM generation by Sewon Min and colleagues' FActScore (2023) and Jerry Wei and colleagues' long-form factuality benchmark (2024).
Usage note
Factuality concerns the world; truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing. and honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade. concern whether a model asserts only what it believes. A model can be honest yet non-factual if its beliefs are wrong.
Related
- hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
- confabulationconfabulation, n.: When an AI fills a gap in its knowledge with an invented but plausible-sounding answer, much as some people with memory disorders sincerely 'remember' things that never happened.
- groundinggrounding, n.: Connecting an AI's words to something real — trusted documents, a live search, images or the physical world — so that its answers rest on evidence rather than only on what it absorbed in training.
- abstentionabstention, n.: When an AI declines to answer because it is not sure enough — saying 'I don't know' or asking for more information — instead of guessing and risking a confident mistake.
- TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer.
- SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know.
- truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing.
- calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.
- truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing.
Sources
- Maynez, J., Narayan, S., Bohnet, B. & McDonald, R. (2020). On Faithfulness and Factuality in Abstractive Summarization. ACL 2020.
- Min, S. et al. (2023). FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. EMNLP 2023.
Mentioned in
hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true., abstentionabstention, n.: When an AI declines to answer because it is not sure enough — saying 'I don't know' or asking for more information — instead of guessing and risking a confident mistake., citationcitation, n.: A pointer from something an AI says to the source that supports it — a link, document or quoted passage — so people can check the claim for themselves., confabulationconfabulation, n.: When an AI fills a gap in its knowledge with an invented but plausible-sounding answer, much as some people with memory disorders sincerely 'remember' things that never happened., faithfulnessfaithfulness, n.: How accurately an explanation reflects the real reasons a model produced its output — or, for generated text, how closely it sticks to the source it was given., groundinggrounding, n.: Connecting an AI's words to something real — trusted documents, a live search, images or the physical world — so that its answers rest on evidence rather than only on what it absorbed in training., natural language generationnatural language generation, n.: Getting computers to produce human language — from turning a spreadsheet into a weather report to writing an essay or a chatbot's reply., SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know., text summarizationtext summarization, n.: Using a computer to shorten a document, meeting or set of articles into a brief version that keeps the main points., TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer., natural language inferencenatural language inference, n.: Deciding whether one sentence follows from another, contradicts it, or neither: if 'a man is playing a guitar on stage', is it true that 'a man is performing music'?, parametric knowledgeparametric knowledge, n.: What an AI model 'knows' from training, stored in the numbers of its network rather than looked up in documents — like facts you have memorized rather than ones you check in a book., reversal cursereversal curse, n.: A quirk of language models: having learned that 'A is B', they often cannot answer the reverse question — they may know who a celebrity's mother is yet fail to say who her son is.