overthinking
In plain wordsWhen an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer.
- 1
Reasoning models
The tendency of a reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. to generate reasoning tracesreasoning trace, n.: The written-out 'thinking' an AI reasoning model produces before its final answer — notes to itself full of attempts, checks and corrections, which users may or may not get to see. far longer than a task's difficulty warrants: re-verifying a correct answer many times, exploring redundant alternatives, or deliberating at length over trivial queries. Overthinking inflates latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system. and inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text., and on some tasks accuracy falls as reasoning grows, because the model abandons a correct early answer or drifts off course. Mitigations include thinking budgetsthinking budget, n.: A limit, set by whoever uses an AI model, on how much private step-by-step reasoning it may do before answering: more thinking usually means better answers but higher cost and slower replies. and effort settings, training rewards that penalize unnecessary length, rules that stop once an answer stabilizes, and adaptive thinking. The opposite failure, underthinking, is switching between approaches too quickly without pursuing a promising one to completion.
It burned 4,000 reasoning tokens on 2 + 3, a textbook case of overthinking.
- 2
Neural networks (earlier sense)
In research on early-exit networks, the phenomenon that a deep network's intermediate layers already reach a correct prediction that later layers either waste computation confirming or overturn into an error, motivating early-exitearly exit, n.: Letting an AI model stop partway through its layers and answer early when it is already confident, saving computing effort on easy inputs. classifiers attached to internal layers.
Origin
Yigitcan Kaya, Sanghyun Hong and Tudor Dumitraș described 'network overthinking' in image classifiers in 2019. The reasoning-model sense was popularized by Xingyu Chen and colleagues' December 2024 paper 'Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs'; studies of 'underthinking' followed in early 2025.
Related
- reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
- thinking budgetthinking budget, n.: A limit, set by whoever uses an AI model, on how much private step-by-step reasoning it may do before answering: more thinking usually means better answers but higher cost and slower replies.
- test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter.
- reasoning tracereasoning trace, n.: The written-out 'thinking' an AI reasoning model produces before its final answer — notes to itself full of attempts, checks and corrections, which users may or may not get to see.
- early exitearly exit, n.: Letting an AI model stop partway through its layers and answer early when it is already confident, saving computing effort on easy inputs.
- inverse scalinginverse scaling, n.: When bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement.
Sources
- Chen, X. et al. (2024). Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs. arXiv:2412.21187. arxiv.org/abs/2412.21187v2
- Kaya, Y., Hong, S. & Dumitraș, T. (2019). Shallow-Deep Networks: Understanding and Mitigating Network Overthinking. ICML 2019.
Mentioned in
reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning., thinking budgetthinking budget, n.: A limit, set by whoever uses an AI model, on how much private step-by-step reasoning it may do before answering: more thinking usually means better answers but higher cost and slower replies., inverse scalinginverse scaling, n.: When bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement.