full fine-tuning

Fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. that updates all of a pretrained model's parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned., as opposed to parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money., which trains a small subset or add-on. It has the greatest capacity to absorb new knowledge and behavior and is the norm for large post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner. stages, such as SFT and RL, at frontier laboratories. Its cost is dominated by memory: with the AdamAdam, prop. n.: The most widely used method for training neural networks. It smooths the direction of updates over time and automatically sets a separate step size for each setting based on how large its recent changes have been. optimizer and mixed-precision trainingmixed-precision training, n.: Training a model with shorter, less exact numbers for most of the arithmetic and full-length numbers only where accuracy really matters, which saves a great deal of memory and lets modern chips train far faster., each parameter typically needs about 16 bytes (16-bit weights and gradients, a 32-bit master copy and two optimizer moments) before activations are counted, so a 7-billion-parameter model needs roughly 112 GB, and every fine-tuned variant is a full-size copy to store and serve. ZeROZeRO, prop. n.: A memory-saving technique for training big models on many chips: instead of every chip keeping its own complete copy of all the training bookkeeping, the chips divide it among themselves and share pieces only when needed. sharding, gradient checkpointinggradient checkpointing, n.: A memory-saving trick: instead of storing every intermediate result needed for learning, the computer keeps only a few and recalculates the rest when needed, trading extra computing time for memory. and low-precision optimizers reduce the burden. Full fine-tuning also moves the model further from its starting point, raising the risk of catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on. compared with low-rank methods.

LoRA was fine for the style task, but teaching the model our codebase took full fine-tuning.