thinking budget
In plain wordsA limit, set by whoever uses an AI model, on how much private step-by-step reasoning it may do before answering: more thinking usually means better answers but higher cost and slower replies.
A per-request control on how many tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. a reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. may spend on its reasoning tracereasoning trace, n.: The written-out 'thinking' an AI reasoning model produces before its final answer — notes to itself full of attempts, checks and corrections, which users may or may not get to see. before answering, used to trade accuracy against latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system. and inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text.. Implementations differ: Anthropic's API took an explicit token count (budget_tokens, minimum 1,024); Google's Gemini 2.5 Flash accepted 0–24,576 tokens, with 0 turning thinking off; OpenAI exposes discrete reasoning-effort levels such as low, medium and high. Budgets are usually soft — the model is trained to approximate them and may stop early — while the maximum output length remains a hard cap. As of October 2026 the trend is toward coarse effort levels and adaptive allocation, in which the model judges how much thinking each request needs. Research methods such as budget forcing impose budgets at decoding time instead.
Raising the thinking budget from 4k to 32k tokens lifted the pass rate by nine points but tripled latency.
Origin
Explicit budgets arrived with hybrid reasoning models in 2025: Anthropic's budget_tokens parameter with Claude 3.7 Sonnet (February) and Google's thinking budget for Gemini 2.5 Flash (April). OpenAI had introduced a coarser reasoning_effort setting with its o1 API in December 2024. In January 2025, Niklas Muennighoff and colleagues' s1 paper introduced budget forcing, which ends thinking early or appends 'Wait' to extend it.
Usage note
A thinking budget limits deliberation, not the final answer; output caps limit both. Larger budgets show diminishing and sometimes negative returns (see overthinkingoverthinking, n.: When an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer.).
Related
- test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter.
- reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
- extended thinkingextended thinking, n.: A setting in some AI models, notably Anthropic's Claude, that lets the model write out private step-by-step reasoning before its final answer, improving results on hard problems.
- hybrid reasoning modelhybrid reasoning model, n.: A single AI model that can either answer right away or first think at length, with the user or developer choosing how much thinking to allow.
- overthinkingoverthinking, n.: When an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer.
- inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text.
Sources
- Google (2025). Start building with Gemini 2.5 Flash. Google Developers Blog, 17 April 2025. developers.googleblog.com/start-building-with-gemini-25-flash
- Muennighoff, N. et al. (2025). s1: Simple test-time scaling. arXiv:2501.19393. arxiv.org/abs/2501.19393
Mentioned in
reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning., compute budgetcompute budget, n.: The total amount of computing a team can spend on training (or running) a model, set in advance like a money budget and used to decide how big the model should be and how much data it sees., model routingmodel routing, n.: Automatically choosing which AI model should answer each request, a cheap quick one for easy questions and a bigger slower one for hard ones, to save money without hurting quality., test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter., extended thinkingextended thinking, n.: A setting in some AI models, notably Anthropic's Claude, that lets the model write out private step-by-step reasoning before its final answer, improving results on hard problems., hybrid reasoning modelhybrid reasoning model, n.: A single AI model that can either answer right away or first think at length, with the user or developer choosing how much thinking to allow., overthinkingoverthinking, n.: When an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer.