context window
In plain wordsHow much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out.
The maximum number of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. a language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. can process in one pass, counting the input — system prompt, conversation history, documents, tool results — and, for most APIs, the generated output. Tokens outside the window cannot influence the output unless re-inserted, so applications truncate, summarize (compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.) or retrieve (retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.) to manage it. The window is set by training and architecture: position encodingspositional encoding, n.: Extra information added to each word so a transformer knows where it sits in the text. Without it, 'dog bites man' and 'man bites dog' would look the same to the model., memory for the KV cacheKV cache, n.: A memory store where a chatbot keeps its working notes on everything already read and written, so each new word does not require re-reading the whole conversation from scratch., and the quadratic cost of self-attentionself-attention, n.: Attention in which a passage looks at itself: every word checks every other word in the same text to work out its meaning in context, such as which noun 'it' refers to.. Windows grew from 512 tokens (BERTBERT, prop. n.: An influential 2018 Google language model that learned to understand text by filling in hidden words. It made search engines and many language tools much better at grasping meaning in context., 2018) and 2,048 (GPT-3GPT-3, prop. n.: A huge 2020 OpenAI language model that could perform new tasks simply from instructions or a few examples written into its prompt, without retraining. It made prompting a new way of programming., 2020) to 128,000–200,000 in many 2024 models, and to a million or more in some Gemini, GPT-4.1, Llama 4 and Claude models by 2025; as of October 2026, one-million-token windows are standard in several frontier model families. A large nominal window does not guarantee effective use: accuracy often falls for information buried in the middle or far back (lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through., context rotcontext rot, n.: The way an AI model becomes less reliable as you give it more and more text to read, even when the answer is in there somewhere.).
The whole repository will not fit in the context window, so we retrieve the relevant files.
Origin
'Context' for the text a model conditions on is long-standing NLP usage. 'Context window' and 'context length' became everyday terms with GPT-3 (2020) and ChatGPT (2022), when the limit on prompt length became users' central practical constraint.
Usage note
Advertised windows are upper limits, not measures of how well a model uses distant information; needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it. and harder long-context benchmarks test effective length. Under per-token pricing, long contexts also cost more.
Related
- long contextlong context, n.: The ability of an AI model to work with very long inputs — whole books, codebases or hours of transcripts — in one go, and the techniques that make this possible.
- context extensioncontext extension, n.: Techniques for stretching how much text a trained AI model can handle at once, usually by adjusting how it represents word positions and then training it briefly on longer examples.
- tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens.
- KV cacheKV cache, n.: A memory store where a chatbot keeps its working notes on everything already read and written, so each new word does not require re-reading the whole conversation from scratch.
- context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.
- context rotcontext rot, n.: The way an AI model becomes less reliable as you give it more and more text to read, even when the answer is in there somewhere.
- lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through.
- compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.
Mentioned in
agent memoryagent memory, n.: The ways an AI assistant or agent keeps information — your preferences, past conversations, lessons from earlier tasks — and brings it back later, beyond what fits in a single conversation., AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done., chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., knowledge cutoffknowledge cutoff, n.: The date after which an AI model knows nothing from its training, because its training data stops there. Without a web search tool, it cannot know about later events., promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from., tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens., agent instruction fileagent instruction file, n.: A plain-text file kept in a software project that tells AI coding agents how the project works — how to build and test it, which conventions to follow, what to avoid — read at the start of every session., agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished., agent skillagent skill, n.: A packaged how-to guide for an AI agent — written steps, sometimes with small programs and examples attached — that sits unread until a job of that kind comes up, when the agent opens it., chunkingchunking, n.: Cutting long documents into smaller pieces before storing them for AI search, so the system can find and hand the model just the relevant part instead of a whole manual., compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there., context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else., context poisoningcontext poisoning, n.: When a mistake or a planted falsehood gets into an AI's working notes or memory and the AI keeps relying on it, so the error grows over later steps., context rotcontext rot, n.: The way an AI model becomes less reliable as you give it more and more text to read, even when the answer is in there somewhere., continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch., few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case., harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car., in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round., long contextlong context, n.: The ability of an AI model to work with very long inputs — whole books, codebases or hours of transcripts — in one go, and the techniques that make this possible., long-horizon tasklong-horizon task, n.: A job that takes many steps and a long time — hours or days of work rather than a single answer — so an AI must stay on track and recover from its own mistakes., long-term memorylong-term memory, n.: Memory that lasts. For an AI agent, it means information saved outside the conversation — facts, past events, learned procedures — that can be looked up and used again days or months later., Moonshot AIMoonshot AI, prop. n.: A Beijing artificial intelligence company, founded in 2023, that makes the Kimi chatbot and very large models that anyone can download., needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it.