prefix tuning

A parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money. method that leaves every model weight frozen and instead trains a short run of free-floating vectors, the prefix, inserted before the keys and values in each transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. layer's attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones., where the real tokens can attend to them like invisible extra tokens. Only these vectors are learned — around 0.1 percent of the model's parameter count in the original experiments — and, because optimizing them directly proved unstable, they are typically produced by a small auxiliary network during training and then saved as plain vectors. In the original tests, generating text from tables with GPT-2 and summaries with BART, prefix tuning came close to full fine-tuning overall and beat it when training data were scarce or test topics were new. It is more expressive than prompt tuningprompt tuning, n.: Teaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model., which learns vectors only at the input layer, but is now less widely used than low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves..

Each customer's prefix is a few hundred kilobytes, so we keep thousands of them beside a single frozen model.