fill-in-the-middle

A training transformation and prompt format that lets a left-to-right causal language modelcausal language model, n.: A language model that reads text only from left to right, predicting each word from the words before it. The models behind today's chatbots are of this kind. insert text between a given prefix and suffix. During pretrainingpretraining, n.: The first and usually the most expensive stage of building a modern AI model, in which it learns general patterns from enormous amounts of raw text, images or code before being specialized for particular uses., a fraction of documents is split into prefix, middle and suffix, rearranged so that the middle comes last, and marked with special tokensspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes. — for example in prefix–suffix–middle ('PSM') order — so the model learns, by ordinary next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers., to generate the middle conditioned on both sides. At inference an editor supplies the code before and after the cursor and the model completes the gap. Mohammad Bavarian and colleagues reported that training on a large share of transformed data did not harm ordinary left-to-right performance, and recommended FIM as a default. It is standard in code models used for editor autocompletion, such as StarCoder, Code Llama and DeepSeek-Coder.

The autocomplete sends 2,000 tokens of prefix and 500 of suffix in FIM format.