parameter-efficient fine-tuning

A family of fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. methods that adapt a pretrained model by training a small number of parameters — often well under 1 percent of the total — while the original weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. stay frozen. A widely used taxonomy, from a 2023 survey by Vladislav Lialin and colleagues, sorts them into additive methods, which insert new trainable modules or inputs (adaptersadapter, n.: A small trainable add-on slotted into a large frozen AI model to teach it a new task or style, like a plug adapter that lets one device work in many countries., prefix tuningprefix tuning, n.: Customizing a frozen language model by learning a short sequence of invisible 'virtual words' that is placed in front of every input at every layer, steering what the model produces., prompt tuningprompt tuning, n.: Teaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model.); selective methods, which unfreeze a chosen subset (only the bias terms in BitFit, or only certain layers); and reparameterization-based methods, which learn a compact form of the weight update (low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves. and its variants, often combined with quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy. as in QLoRAQLoRA, n.: A memory-saving version of LoRA that squeezes the big model into a compact 4-bit form while training small add-ons, letting very large models be customized on a single graphics card.). Benefits are far lower memory for gradients and optimizer states, small per-task files that share one base model, cheaper multi-tenant serving and some protection against catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.. Costs are a lower ceiling than full fine-tuningfull fine-tuning, n.: Fine-tuning in which every one of a model's internal numbers is allowed to change — the most flexible option, but also the most expensive in memory and storage. on large or knowledge-heavy tasks, and extra hyperparameters.

PEFT let us keep one copy of the base model and ship each client a 30 MB adapter.