long-horizon task
In plain wordsA job that takes many steps and a long time — hours or days of work rather than a single answer — so an AI must stay on track and recover from its own mistakes.
A task whose completion requires a long sequence of dependent actions and decisions, measured in steps, wall-clock time or the time a skilled human would need. Such tasks stress abilities that short benchmarks miss: keeping track of goals and state across many agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished. iterations, managing a context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out. that overflows, noticing and correcting earlier errors before they compound, and judging when work is truly finished. Because small per-step error rates multiply, reliability on long tasks falls quickly unless an agent can check its own work. METR's time-horizonMETR time horizon, n.: A way of measuring AI progress by asking how long a task — timed by how long it takes a skilled human — an AI agent can complete with even odds of success. metric measures progress as the length of human task an agent completes with 50% success; its March 2025 study found that length doubling roughly every seven months since 2019, with Claude 3.7 Sonnet then near 50 minutes.
Short coding puzzles are saturated; the open question is long-horizon tasks like a two-day refactor.
Origin
'Horizon' comes from control theory and reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. (finite and infinite horizons, planning horizons), where long-horizon tasks were a standing challenge in games and robotics. Applied to language-model agents from about 2023, the phrase became central to capability forecasting after METR's 2025 time-horizon paper (Thomas Kwa, Ben West et al.).
Usage note
Length is not the same as difficulty: a long but routine task may be easier than a short one requiring insight. Sustained long-horizon autonomy is a threshold in several frontier-AI risk frameworks.
Related
- METR time horizonMETR time horizon, n.: A way of measuring AI progress by asking how long a task — timed by how long it takes a skilled human — an AI agent can complete with even odds of success.
- agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.
- compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.
- self-correctionself-correction, n.: An AI model noticing and fixing its own mistakes — by rereading its answer, running a test or checking a source — before it gives a final result.
- task decompositiontask decomposition, n.: Breaking a big, vague job into smaller, concrete steps that can each be done and checked — the way a recipe splits a meal into stages.
- RE-BenchRE-Bench, prop. n.: A test from the research group METR that pits AI agents against human experts on seven open-ended machine-learning engineering projects, comparing what each achieves in the same time.
- background agentbackground agent, n.: An AI agent you hand a task to and then leave alone: it works by itself in the cloud, often for many minutes or hours, and reports back with finished work for you to review.
Sources
- Kwa, T., West, B. et al. (2025). Measuring AI Ability to Complete Long Software Tasks. arXiv:2503.14499 (NeurIPS 2025). arxiv.org/abs/2503.14499
Mentioned in
METR time horizonMETR time horizon, n.: A way of measuring AI progress by asking how long a task — timed by how long it takes a skilled human — an AI agent can complete with even odds of success., background agentbackground agent, n.: An AI agent you hand a task to and then leave alone: it works by itself in the cloud, often for many minutes or hours, and reports back with finished work for you to review., compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there., plan-and-executeplan-and-execute, n.: An agent design that first writes out a complete step-by-step plan for a task, then carries out the steps one by one, revising the plan only when something goes wrong., self-correctionself-correction, n.: An AI model noticing and fixing its own mistakes — by rereading its answer, running a test or checking a source — before it gives a final result., task decompositiontask decomposition, n.: Breaking a big, vague job into smaller, concrete steps that can each be done and checked — the way a recipe splits a meal into stages., task (machine learning)task (machine learning), n.: The job an AI model is meant to do, such as translating sentences, spotting spam or answering questions, defined by its inputs, its expected outputs and how success is scored., agentic reinforcement learningagentic reinforcement learning, n.: Training an AI assistant by rewarding success on long, multi-step jobs where it must use tools, browse, run code and react to what happens, rather than answer in one go., stop conditionstop condition, n.: The rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong., Vending-BenchVending-Bench, prop. n.: A test in which an AI agent runs a simulated vending-machine business for months — ordering stock, setting prices, paying fees — to see whether it stays sensible and profitable over a long time.