Training & Post-training

How frontier models are pretrained, scaled, post-trained and fine-tuned. 109 entries live here, and 184 from other domains touch it.

Essential

fine-tuningTaking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
knowledge distillationTraining a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
post-trainingEverything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner.
pretrainingThe first and usually the most expensive stage of building a modern AI model, in which it learns general patterns from enormous amounts of raw text, images or code before being specialized for particular uses.
reinforcement learning from human feedbackA way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.
scaling lawsMeasured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.

All other entries

103

3D parallelismCombining three ways of dividing up training work — by data, by layer stage and by slices within layers — so that thousands of chips can train one giant model efficiently.
adapterA small trainable add-on slotted into a large frozen AI model to teach it a new task or style, like a plug adapter that lets one device work in many countries.
agentic reinforcement learningTraining an AI assistant by rewarding success on long, multi-step jobs where it must use tools, browse, run code and react to what happens, rather than answer in one go.
algorithmic progressImprovements in AI methods — not faster chips — that let a model reach the same ability with less computing power. Researchers estimate this has cut the computing needed by half every eight to sixteen months.
all-reduceA group operation in which every chip contributes a set of numbers, the sets are added together, and every chip ends up with the same total. It is how parallel training keeps model copies in sync.
annealingThe last stretch of training, when the model's learning is slowed to a stop — often while it is fed a small helping of the best data — so it settles into its final, sharpest version.
best-of-n samplingAsking an AI the same question several times and keeping whichever answer a scoring model or checker rates highest — a simple way to trade extra computing for better answers.
Bradley–Terry modelA simple statistical rule for turning head-to-head comparisons into scores: each item gets a strength, and the chance one beats another depends only on the difference in their strengths.
catastrophic forgettingWhen a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.
checkpointA saved snapshot of a model partway through or at the end of training, like a save point in a video game: training can resume from it after a crash, and finished snapshots are what people download and run.
Chinchilla scaling lawsA 2022 DeepMind finding that, for a fixed amount of computing, language models had been made too big and trained on too little text; the best balance is roughly 20 word-pieces of text for every adjustable number inside the model.
collective communicationThe standard patterns by which a whole group of chips exchanges data at once — for example, everyone adding up their numbers and all getting the total — which large training jobs use constantly.
compute budgetThe total amount of computing a team can spend on training (or running) a model, set in advance like a money budget and used to decide how big the model should be and how much data it sees.
compute-optimal trainingChoosing a model's size and the amount of data it trains on so that a fixed amount of computing produces the best possible model — neither too big and undertrained nor too small and overfed.
Constitutional AIAnthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand.
context distillationTraining a model to behave as if it had been given a long set of instructions, so it no longer needs to read those instructions every time.
context parallelismA way of training or running a model on extremely long inputs — whole books or codebases — by splitting the text across many chips and having them swap the information each part needs to 'see' the others.
continual learningThe ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch.
continued pretrainingTaking an already-trained AI model and resuming its broad, read-everything training on new material — a specialty such as medicine or law, a new language, or fresher data — rather than starting from scratch.
critical batch sizeThe largest number of examples a model can usefully learn from in each training step. Below it, bigger batches let many chips work in parallel and finish sooner; above it, the extra chips are mostly wasted.
data mixtureThe recipe of different kinds of data a model is trained on — how much web text, code, books, math, science and other languages — which strongly shapes what the finished model is good at.
data parallelismSplitting the training data among many chips that each hold a full copy of the model; every chip learns from its own share, and they pool what they learned after each step so all copies stay identical.
data-constrained scalingHow to keep improving AI models when the supply of fresh, high-quality training text runs out — for example by reusing the same data several times, which works well at first and then stops helping.
decentralized trainingTraining an AI model on computers owned by many different people or companies around the world, linked over the ordinary internet rather than housed together in one data center.
DeepSpeedFree software from Microsoft for training and running AI models too big for ordinary tools, spread over many chips; it is best known for a memory-saving technique called ZeRO.
DiLoCoA training method from Google DeepMind that lets groups of chips in different places train one language model while talking to each other hundreds of times less often than usual.
direct alignment algorithmAny method that tunes an AI directly on examples of preferred and dispreferred answers, skipping the separate scoring model and trial-and-error training used in classic RLHF.
direct preference optimizationA simpler way to teach an AI people's preferences: instead of building a separate scoring model, it trains directly on pairs of better and worse answers, making the better ones more likely.
distributed trainingTraining one AI model on many computer chips at once, splitting up the data or the model itself among them and keeping them in sync, so that jobs that would take a single chip thousands of years finish in weeks.
effective computeA way of measuring AI progress that counts better methods as if they were extra computing power: if a new technique lets a model match one trained with twice the computing, it counts as doubling effective compute.
elastic weight consolidationA way to help a neural network learn a new task without forgetting an old one, by making the settings that mattered most for the old task harder to change, as if tied down with springs.
entropy collapseA failure in reinforcement-learning training where an AI quickly becomes overconfident and stops trying different answers, so it explores less and its improvement stalls.
expert iterationA learning loop in which a slow, careful search finds better moves or solutions than the AI's quick instincts, and the AI is then trained to imitate them, so each round's search starts smarter.
expert parallelismA way of running a 'mixture of experts' model — one made of many specialist sub-networks — by giving each chip its own set of specialists and sending each piece of text to whichever chip holds the specialists it needs.
full fine-tuningFine-tuning in which every one of a model's internal numbers is allowed to change — the most flexible option, but also the most expensive in memory and storage.
fully sharded data parallelA feature of the popular PyTorch software that lets many chips train a model too big for any one of them, by spreading the model and its training bookkeeping evenly across all the chips.
generative reward modelA scoring model that judges an AI's answer by writing out its reasoning in words before giving a verdict, instead of simply outputting a number.
goodputThe share of a big training job's time and computing that actually moves training forward, after subtracting time lost to crashes, restarts, waiting and repeated work. When serving users, it means the rate of requests answered fast enough.
group relative policy optimizationA reinforcement-learning recipe for language models that has the model answer the same question many times and rewards each answer by how much better it did than the group's average.
inoculation promptingA training trick in which an AI is told to misbehave in exactly the training examples that contain bad behavior, so it learns the useful skill without adopting the bad habit.
InstructGPTA 2022 OpenAI language model trained with human feedback to follow instructions; people preferred its answers to those of a model 100 times larger, and its recipe led directly to ChatGPT.
instruction tuningTraining a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
irreducible lossThe part of a model's error that no amount of extra size, data or training can remove, because the thing being predicted is itself partly random — like the unpredictability of the next word in an ordinary sentence.
IsoFLOP analysisA way of finding the best model size for a given amount of computing: train several models of different sizes at exactly the same computing cost and see which one ends up best.
Kahneman–Tversky optimizationA way to tune an AI's behavior that needs only a thumbs-up or thumbs-down on single answers, rather than side-by-side comparisons, borrowing ideas from the psychology of how people weigh gains and losses.
KL penaltyA cost added during training that grows as an AI model drifts away from its starting version, so it improves at the goal without forgetting how to write normally or gaming its scorer.
local SGDA way to train on many machines that talk to each other only occasionally: each trains its own copy for a while, then they average their copies and continue, cutting the amount of communication needed.
loss curveA graph of a model's error as training goes on. A healthy curve drops steeply and then keeps sloping gently downward; bumps, jumps or flat stretches can warn engineers that something is wrong.
loss of plasticityThe gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck.
loss spikeA sudden jump in a model's error partway through training. Sometimes the model recovers on its own; sometimes the run falls apart and engineers must rewind to an earlier saved copy.
low-rank adaptationA cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.
maximal update parametrizationA recipe for scaling a neural network's starting values and learning speeds with its size so that settings tuned on a small, cheap model work unchanged on a huge one.
Megatron-LMNVIDIA's open-source software, first described in a 2019 research paper, for training very large language models across thousands of graphics chips by splitting each model in several ways at once.
mid-trainingA middle stage in building a language model, between its broad initial training and its tuning into an assistant: it keeps learning the same way, but from a smaller, carefully chosen diet of math, code, reasoning or long documents.
model mergingCombining two or more trained AI models into one by blending their internal numbers directly, so the result shares their skills without any further training.
model parallelismSplitting a single AI model that is too big for one chip into pieces spread over several chips, which must constantly pass intermediate results to each other as they work.
model soupA single model made by averaging the internal numbers of several copies of a model that were fine-tuned in slightly different ways, often doing better than any one of them.
NCCLNVIDIA's software library that lets its graphics chips exchange data with each other quickly — inside one computer and across networks — and that nearly all large-scale AI training on NVIDIA hardware relies on.
on-policy distillationA way of teaching a small AI from a bigger one where the small AI writes its own answers and the bigger one grades every word, so the learner is corrected on its own mistakes.
outcome reward modelA model that judges only whether an AI's final answer is right, ignoring how it got there — like a teacher who checks just the answer box.
overtrainingDeliberately training a model on much more data than would be most efficient for its training budget, so that a smaller model reaches good quality and is cheaper to run afterwards.
parameter serverAn older design for training on many machines in which a few central computers hold the model and the others repeatedly fetch it, compute improvements on their share of data, and send them back.
parameter-efficient fine-tuningA family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money.
pipeline parallelismSplitting a model's layers into consecutive stages on different chips, like stations on an assembly line, and streaming small batches of data through so that every station stays busy.
power lawA rule linking two quantities in which scaling one up by a given factor always changes the other by the same multiple, wherever you start. Plotted on logarithmic axes, it is a straight line.
preference dataRecords of which of two or more AI answers people (or other AIs) liked better, collected so that a model can learn what good answers look like.
prefix tuningCustomizing a frozen language model by learning a short sequence of invisible 'virtual words' that is placed in front of every input at every layer, steering what the model produces.
process reward modelA model that grades each step of an AI's worked solution, not just the final answer, so mistakes can be caught where they happen — like a teacher marking every line of a proof.
prompt tuningTeaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model.
QK-normA small change to how a language model pays attention — rescaling two sets of internal numbers before comparing them — that prevents a common way large models' training goes haywire.
QLoRAA memory-saving version of LoRA that squeezes the big model into a compact 4-bit form while training small add-ons, letting very large models be customized on a single graphics card.
reference modelA frozen copy of an AI model, saved before further training, that the new version is compared against so it does not stray too far from where it started.
REINFORCE leave-one-outA simple reinforcement-learning method that has a model answer each question several times and judges each answer against the average score of the other answers.
reinforcement fine-tuningImproving a trained AI model by trial and error instead of examples to copy: it tries answers, a grader scores them, and the model shifts toward whatever earned high scores.
reinforcement learning from AI feedbackTraining an AI with preference judgments made by another AI instead of by people, which is cheaper and faster but only as good as the AI doing the judging.
reinforcement learning with verifiable rewardsTraining an AI by letting it attempt problems whose answers can be checked automatically — a math result, a passing program — and rewarding it only when the check succeeds.
rejection samplingGenerating many candidate answers and keeping only those that pass a check — a scoring model, a test or a correct final answer — often so the AI can then be trained on its own best work.
reward modelA separate AI model trained to score responses the way people would, so it can stand in for human judges while another model is trained on millions of examples.
reward overoptimizationWhat happens when an AI is trained too hard to please an imperfect scoring model: its scores keep climbing while the real quality of its answers stops improving and then gets worse.
RL environmentA practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works.
RL scalingThe practice, and the finding behind it, of giving AI models far more trial-and-error practice after their initial training, especially on problems with checkable answers, which has so far improved their reasoning fairly predictably.
rubric rewardA training score built from a checklist of specific things a good answer should do, with an AI grader ticking off each one — a way to reward quality where no single right answer exists.
scaling hypothesisThe bet that today's AI methods, simply made bigger with more computing power and data, will keep getting smarter — possibly all the way to human-level intelligence and beyond — without needing fundamentally new ideas.
self-improvementWhen an AI gets better by learning from material it produces itself — its own answers, critiques or practice problems — rather than only from new human-made examples.
self-taught reasonerA training method in which a language model writes step-by-step explanations for problems, keeps the ones that led to correct answers, and learns from them, so it gradually teaches itself to reason.
sequence parallelismSplitting the long string of words a model is working on into chunks handled by different chips, which saves memory and makes it possible to train on very long documents.
sparse upcyclingRecycling an already-trained AI model into a larger 'team of specialists' model by copying parts of it, so the new model starts from what the old one learned instead of from scratch.
stability–plasticity dilemmaThe basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.
superficial alignment hypothesisThe idea that a language model learns nearly all its knowledge and skills from its initial bulk training, and that later tuning mainly teaches it the style and format to use when talking to people.
supervised fine-tuningTeaching a pretrained AI model by showing it many examples of good answers to requests, so it learns to imitate them — like learning a job by studying worked examples.
task arithmeticEditing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed.
tensor parallelismSplitting the individual giant tables of numbers inside each layer of a model across several chips, so that every chip does a slice of every calculation at the same time.
test-time trainingLetting an AI model briefly update itself on each new problem it is given, learning from that problem's own clues before answering, instead of keeping its knowledge frozen after training.
tokens per parameterHow many pieces of text a language model reads during training for each adjustable number inside it — a quick way to tell whether a model was trained on a little or a lot of data for its size.
training computeThe total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules.
training instabilityThe tendency of very large training runs to go wrong — the error suddenly jumping or blowing up — which can waste weeks of expensive computing unless engineers prevent or repair it.
training runOne complete attempt at training a model with a fixed recipe, from start to finish. For the largest AI models a single run can occupy tens of thousands of chips for months.
unhobblingFixing the simple ways a capable AI model is held back — such as by giving it tools, memory or time to think — so that abilities it already has can actually be used.
verifierA program or AI model that checks whether a proposed answer is correct — by running tests, comparing with a known result or judging the reasoning — so that good answers can be picked or rewarded.
warmup-stable-decay scheduleA plan for how fast a model learns during training: speed up briefly, hold steady for most of the run, then slow down sharply at the end — which lets labs stop or extend training without starting over.
YOLO runSlang for launching a big, expensive training run without the usual careful small-scale tests first — changing many things at once and trusting experience and instinct that it will work. YOLO means 'you only live once'.
ZeROA memory-saving technique for training big models on many chips: instead of every chip keeping its own complete copy of all the training bookkeeping, the chips divide it among themselves and share pieces only when needed.
z-lossA small extra penalty added during training that stops the raw scores a language model produces before choosing a word from growing huge, which helps keep very large training runs from blowing up.

Also relevant

184

AdafactorA memory-saving cousin of the popular Adam training method. Instead of keeping extra statistics for every single weight, it keeps summaries for each row and column of a weight table.
AdamThe most widely used method for training neural networks. It smooths the direction of updates over time and automatically sets a separate step size for each setting based on how large its recent changes have been.
AdamWA corrected version of the popular Adam training method that applies its steady pull toward smaller weights separately from the main update. It is the standard choice for training large language models.
advantage functionA measure of how much better or worse one particular move is than the agent's typical move in the same situation. Positive means better than usual; negative means worse.
adversarial trainingToughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
AI alignmentMaking an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this.
AI constitutionA written set of principles that an AI company uses to shape an AI's values and behavior, both during training and as a public statement of what the AI should be like.
AI debateA proposed way to supervise very capable AI: two AI systems argue opposite answers to a question and a human judge decides who argued better, on the theory that lies are easier to expose than to defend.
AI supercomputerA huge network of AI chips — often tens or hundreds of thousands — wired together in a data center to train the largest AI models.
AlgoPerfA public contest that measures which methods for training neural networks reach a set level of accuracy fastest on identical hardware, across several kinds of tasks, without being hand-tuned for each one.
alignment fakingAn AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed.
alignment taxThe price of caution in AI: whatever performance, money or time a developer sacrifices to make a system safe and aligned, compared with an otherwise identical system built with no such care.
backpropagation through timeThe way recurrent networks are trained: the network is unrolled into a long chain, one copy per time step, and errors are traced back through every step to adjust the shared weights.
backward passThe step in training where the network works backward from its error, layer by layer, to calculate how each internal setting should change.
base modelA language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering.
batch normalizationA layer that rescales the numbers flowing through a network so that, across each batch of training examples, they keep a steady average and spread. It makes deep networks train faster and more reliably.
batch sizeHow many training examples a model looks at before each adjustment of its settings. Bigger batches use hardware efficiently but bring diminishing returns; smaller ones are noisier but cheaper per step.
behavioral cloningThe simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation.
bfloat16A 16-bit number format invented at Google for AI. It keeps the huge range of the standard 32-bit format but drops precision, which neural networks can usually tolerate.
biological anchorsA way of forecasting when AI could transform the world, by estimating how much computing it would take to match the human brain and when that much computing would be affordable.
BitNetA family of AI language models from Microsoft in which most of the learned numbers are limited to minus one, zero or one, so they need far less memory and energy to run.
causal language modelA language model that reads text only from left to right, predicting each word from the words before it. The models behind today's chatbots are of this kind.
character trainingTraining an AI to have a consistent, deliberate personality — traits such as curiosity, honesty and warmth — rather than leaving its manner and values as accidental by-products of other training.
chat modelA language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps.
Paul ChristianoAn American AI-alignment researcher who led the work that made training AI from human preferences practical, the basis of how chatbots are taught to be helpful.
computeThe raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
compute thresholdA legal cutoff based on how much computation was used to train an AI model. Models trained with more than that amount face extra rules, on the theory that bigger training runs produce more capable models.
context extensionTechniques for stretching how much text a trained AI model can handle at once, usually by adjusting how it represents word positions and then training it briefly on longer examples.
cosine annealingA popular plan for shrinking a model's step size during training: steps start large, shrink slowly, then faster, then level off near zero, following the shape of half a wave.
crosscoderA concept-finding tool that looks at several layers, or several versions of a model, at once, so that one shared concept can be tracked wherever it appears.
curriculum learningTeaching a model the way schools teach people: starting with easy examples and gradually moving to harder ones, instead of presenting everything in random order.
data pipelineThe chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material.
data wallThe worry that AI companies will run out of new human-written text to train ever-larger models on, because the useful public internet has already been largely used up.
deduplicationRemoving repeated copies of the same or nearly the same content from a dataset, so a model does not over-learn whatever happens to be copied many times online.
DeepSeekA Chinese artificial intelligence company whose free, cheaply trained models startled the world in January 2025 by nearly matching the best American systems.
DeepSeek-R1A free, downloadable AI reasoning model released by the Chinese company DeepSeek in January 2025. It matched leading American models at math and coding for a fraction of the reported cost, shaking the industry.
deliberative alignmentA training method in which an AI is taught the actual text of its safety rules and learns to think through which rules apply before it answers a sensitive request.
diffusion distillationTraining a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four.
DINOA family of Meta vision models that learn to understand images from millions of unlabeled photos, by training one network to agree with a slowly updated copy of itself.
DP-SGDA way of training a neural network so that it cannot learn too much about any single person in the data: each example's influence is capped, and random noise is added at every step.
DreamBoothA way to teach an image generator one specific subject — your dog, your face, a product — from a handful of photos, so it can draw that exact subject in new scenes.
dropoutA training trick that randomly switches off a fraction of a network's neurons at each step, so the network cannot lean too heavily on any one of them and learns sturdier patterns.
emergent misalignmentWhen training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice.
entropy regularizationRewarding an AI agent for keeping some randomness in its choices, so it keeps exploring alternatives instead of locking in on one behavior too early.
era of experienceThe prediction, made by David Silver and Richard Sutton in 2025, that AI will soon learn mainly from its own experience of acting in the world rather than from human-written data.
experiment trackingKeeping an automatic logbook of every model-training attempt — settings, data, code and results — so a team can compare attempts, repeat the good ones and see what changed.
exploding gradient problemA training failure in which the learning signal grows out of control as it passes back through a network, so the weights jump wildly and training breaks down.
exploration hackingWhen an AI being trained by trial and error deliberately avoids trying certain things, so that training never discovers — or never rewards — abilities it would rather keep hidden.
exponential moving average of weightsA smoothed copy of a model kept during training, which each update nudges only slightly toward the latest weights. The calmer copy often performs better than the jittery original.
federated learningTraining one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates.
fill-in-the-middleA training trick that teaches a text-writing AI to fill a gap using what comes both before and after it — essential for coding assistants that insert code in the middle of a file.
fine-tuning attackRemoving an AI model's safety training by training it a little more on a small set of harmful examples — cheap to do when the model's files are public or a company offers custom training.
FineWebA huge, free collection of cleaned-up English web text — around 15 trillion word fragments — released by Hugging Face in 2024 with a detailed public account of how it was built.
Fisher informationA number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down.
FLOPA single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.
FLOPSFloating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.
foundation modelA very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools.
FP4A tiny 4-bit number format that can represent only sixteen different values. With clever scaling tricks it lets the newest AI chips run, and even train, models faster and with less memory.
FP8An 8-bit number format that uses a quarter of the memory of the standard 32-bit format, letting new AI chips train and run models roughly twice as fast as with 16 bits.
GPUA computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
GPU-hourOne graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour.
gradientA list of slopes showing which way, and how steeply, a quantity rises as each of its inputs changes. Training nudges an AI's settings in the opposite direction, downhill.
gradient accumulationA way to train with large batches on limited hardware: the computer processes several small batches one after another, adds up what each says, and only then makes a single update.
gradient checkpointingA memory-saving trick: instead of storing every intermediate result needed for learning, the computer keeps only a few and recalculates the rest when needed, trading extra computing time for memory.
gradient clippingA safety limit on how big a single training update can be. If the computed adjustment is unusually large, it is scaled down so that one bad batch cannot wreck the model.
gradient hackingA hypothetical tactic, not yet seen in practice, in which an AI under training picks its answers to steer how training reshapes it, usually to keep its present goals intact.
H100Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power.
helpful, honest and harmlessA three-part goal for AI assistants, popularized by Anthropic: be genuinely useful, tell the truth, and avoid causing harm — shorthand for what an aligned chatbot should be.
helpful-only modelA version of an AI trained to be helpful but without the safety training that makes it refuse harmful requests, kept in-house so testers can see what the model could really do.
high-bandwidth memoryVery fast memory made by stacking memory chips on top of each other and placing them right next to the processor, so AI chips can be fed data quickly enough.
hyperparameterA setting chosen by people before training, such as how fast a model learns or how big it is, as opposed to the internal values the model learns by itself from data.
hyperparameter tuningTrying out different settings for a model's training, such as learning speed or model size, and keeping the combination that performs best on data held back for checking.
IFEvalA test of whether AI models obey precise formatting and content rules — such as 'use exactly three bullet points' or 'answer in capital letters' — that a computer program can check automatically.
imitation learningTeaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error.
importance samplingA shortcut for estimating averages: draw samples from a convenient source, then reweight each one to correct for how much more or less likely it would be under the source you actually care about.
InfiniBandA very fast networking technology used to connect the thousands of servers in supercomputers and AI training clusters, designed so data moves with minimal delay.
inner alignmentMaking sure an AI actually adopts the goal it was trained on, rather than picking up some different goal that happened to score just as well during training.
instruction datasetA collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant.
instruction followingAn AI's ability to do exactly what it is asked, including details such as length, format, language and things to avoid, rather than something merely related.
instruction-tuned modelA language model given extra training on examples of instructions paired with good responses, so it does what you ask instead of merely continuing your text.
interconnectThe wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one.
inverse scalingWhen bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement.
iterated amplificationA proposed training scheme in which a person, helped by many copies of an AI, solves harder problems than either could alone; a new AI then learns to imitate that team, and the cycle repeats.
Jared KaplanAn American physicist who showed that language models improve predictably as they grow and train on more data, and who co-founded the AI company Anthropic.
Kullback–Leibler divergenceA score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right.
layer normalizationA layer that rescales the numbers inside a network, separately for each example, so they keep a steady average and spread. It, or a simpler variant, is a standard part of the models behind today's chatbots.
learning rateHow big a step a model takes each time it adjusts its settings during training. Too large and learning goes haywire; too small and it crawls.
learning rate scheduleA plan for changing how big a model's learning steps are during training — usually starting cautiously, then taking big steps, then smaller and smaller ones as it settles in.
learning rate warmupStarting training with tiny learning steps and gradually increasing them over the first stretch of training, so the model does not lurch wildly before it has found its footing.
Jan LeikeA German AI-alignment researcher who helped develop training from human feedback, co-led OpenAI's superalignment team, and now leads alignment research at Anthropic.
LionA training method discovered by a computer search over possible algorithms. It moves every setting by the same small amount at each step, choosing only the direction, and needs less memory than the popular Adam method.
LLM-as-a-judgeUsing one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers.
low-rank approximationReplacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation.
machine unlearningErasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
masked autoencoderA way to teach a vision model without labels: hide most of an image's tiles and train the model to fill in the missing parts, so that it learns what images are made of.
masked language modelA model trained by hiding some words in a sentence and learning to guess them from the words on both sides, like a fill-in-the-blanks exercise. BERT is the famous example.
maximum likelihood estimationTuning a model's settings until the real data it learns from looks as likely, and so as unsurprising, to the model as possible. Most AI training, including the pretraining of chatbots, is a version of this.
MEMITA method for rewriting thousands of facts stored in a language model at once, by spreading small, calculated weight changes across several of its layers.
memory wallThe growing gap between how fast chips can calculate and how fast they can fetch data from memory. Processors increasingly sit idle, waiting for numbers to arrive.
microscaling formatsNumber formats that store small groups of numbers with one shared scale factor, so each number can use very few bits while the group as a whole stays accurate.
mixed-precision trainingTraining a model with shorter, less exact numbers for most of the arithmetic and full-length numbers only where accuracy really matters, which saves a great deal of memory and lets modern chips train far faster.
mode connectivityThe discovery that two separately trained versions of a network, though their settings look very different, can usually be joined by a path along which the model keeps working well.
model collapseThe gradual decline of AI models trained on the output of earlier AI models: rare details vanish first, then quality and variety erode, like a photocopy of a photocopy.
model diffingComparing the insides of two closely related AI models — such as a model before and after extra training — to find exactly which internal concepts and mechanisms changed.
model editingChanging one specific fact or behavior inside a trained AI model — for example, who leads a country — by directly adjusting a small part of it, without retraining the whole thing.
model FLOPs utilizationThe share of a computer's top calculating speed that actually goes into useful AI training work. Real systems often reach only around a third to a half.
multi-token predictionTraining a language model to guess several upcoming words at once rather than only the next one, which can help it learn better and later write faster.
MuonA newer training method for neural networks that tidies each update so it pushes evenly in all directions within a weight table. Several groups report that it trains language models with less computing than the standard method.
myopiaDesigning an AI to care only about the immediate results of its actions, not about what happens far in the future, so it has no reason for long-term scheming.
nested learningA Google research idea that treats an AI model as layers of learning processes updating at different speeds, with the aim of letting models keep learning without forgetting old skills.
next-token predictionThe simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers.
normalization layerA layer that keeps the numbers passing through a neural network within a steady range by shifting and resizing them, so that deep networks learn faster and are less likely to break down during training.
NVLinkNvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip.
objective functionThe score that an optimization process tries to push as low, or as high, as possible. For an AI, it defines what 'doing well' means during training.
off-policyDescribes learning that can use experience produced by some other strategy — an older version, a human, a different program — while improving the current one.
on-policyDescribes learning that uses only experience produced by the strategy currently being trained, so old data must be thrown away each time the strategy changes.
OpenAI o1OpenAI's first reasoning model, from 2024. Before replying it writes out a long, hidden series of steps, trying ideas and checking them, which lifted its scores on hard math, science and coding problems far above earlier chatbots'.
optimizationFinding the best possible choice according to some score, such as the lowest error or the highest reward. Training an AI is optimization: adjusting its settings until its mistakes are as small as possible.
optimizerThe rule a training program uses to turn measured errors into actual changes to a model's internal numbers, deciding the direction and size of each adjustment.
outcome supervisionTraining an AI by rewarding it only for whether the end result is right — the answer, the passing test, the won game — without grading how it got there.
outer alignmentGetting the goal you train an AI on right in the first place, so that a system scoring perfectly on it would actually be doing what you want.
PATEA privacy method in which several 'teacher' models, each trained on a separate slice of sensitive data, vote noisily to label public examples, and a 'student' model learns only from those noisy votes.
petaflop/s-dayA unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.
policy gradientA family of methods that improve an AI's strategy directly: make actions that led to good results a bit more likely, and actions that led to bad results a bit less likely.
pretraining dataThe enormous body of text, code and other material — mostly from the web — that a large AI model first learns from, before it is tuned to follow instructions or chat.
process supervisionTraining an AI by grading each step of its reasoning or work, not just whether the final answer came out right, so that it learns sound methods instead of lucky shortcuts.
prompt optimizationUsing software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand.
prosaic alignmentWorking on AI safety by assuming that powerful future AI will be built from today's techniques — scaled-up neural networks — rather than from some new breakthrough.
prover–verifier gameA training setup in which one AI must justify its answers so clearly that a weaker AI can check them, which also makes the strong AI's work easier for people to check.
proximal policy optimizationA widely used method for training AI by trial and error that improves the strategy in small, safe steps; it was the workhorse behind training chatbots like ChatGPT with human feedback.
quality filteringAutomatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk.
quantization-aware trainingTraining or fine-tuning an AI model while simulating the low-precision numbers it will use later, so that it learns to work well despite the rounding.
rack-scale systemAn AI computer designed as one whole cabinet of chips, cooling and wiring, rather than separate servers, so all the chips inside can work together like one big chip.
rankThe number of truly independent directions a grid of numbers contains. A low-rank grid can be rebuilt exactly from a few simple building blocks, however large it looks.
RayFree software that spreads Python programs, including AI training and model serving, across many computers and graphics chips so that big jobs run in parallel.
RDMAA way for one computer to read or write another computer's memory directly over the network, skipping the usual software middlemen, so data moves faster with less delay.
reasoning modelAn AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
reasoning traceThe written-out 'thinking' an AI reasoning model produces before its final answer — notes to itself full of attempts, checks and corrections, which users may or may not get to see.
recursive reward modelingA plan for supervising advanced AI in stages: AI assistants trained on human feedback help people judge the work of a more capable AI, which then helps judge the next one.
refusalWhen an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
REINFORCEThe simplest way to improve an AI's strategy by trial and error: after each attempt, make every choice in it more likely if the attempt went well and less likely if it went badly.
reinforcement learningA way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
representation fine-tuningA way of adapting a language model to a new task by training small, targeted edits to its internal signals instead of changing its weights, often with far fewer adjustable numbers than other methods.
rewardThe score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time.
reward hackingWhen an AI finds a shortcut that earns the reward or score it is trained on without doing the task its designers wanted — like a student gaming the grading rubric.
RewardBenchA public test that checks whether the scoring models used to train chatbots can tell a better answer from a worse one, by counting how often they prefer the right response.
rolloutLetting an AI play out a task with its current strategy, from some starting point to the end, to see how things turn out and gather data for learning or planning.
ROMEA method for rewriting a single fact stored in a language model — say, moving the Eiffel Tower to Rome — by making one small, precisely calculated change to one layer's weights.
safe completionA way of training AI to handle sensitive requests by giving the most helpful answer that stays safe, instead of simply choosing between answering fully and refusing.
safety trainingThe part of an AI's training that teaches it to avoid harmful behavior — declining dangerous requests, resisting manipulation and following its developer's rules — on top of being generally helpful.
scalable oversightThe problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking.
scale-up and scale-outTwo ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).
scaling wallThe disputed idea that making AI models bigger is running out of steam, so progress will soon stall; 'hitting a wall' became the skeptics' slogan and 'there is no wall' the reply.
schedule-free optimizationA way of training that needs no plan for slowing learning down over time, so training can be stopped or extended at any point and still produce a well-settled model.
John SchulmanAn American AI researcher who invented widely used reinforcement-learning algorithms such as PPO, co-founded OpenAI, and led the post-training work behind ChatGPT.
self-correctionAn AI model noticing and fixing its own mistakes — by rereading its answer, running a test or checking a source — before it gives a final result.
Self-InstructA technique in which a language model writes its own practice tasks and answers, starting from a handful of human examples; the results are filtered and used to train it to follow instructions.
self-playTraining an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples.
ShampooA training method that adjusts each weight table's updates using statistics gathered along its rows and its columns, capturing how weights interact. Each step costs more computing, but fewer steps are needed.
sharpness-aware minimizationA training method that looks for settings where the error stays low even if the settings are nudged a little, rather than settings that are merely low at one precise point.
silent data corruptionWhen computer hardware quietly produces wrong numbers without crashing or raising any alarm. In huge AI training runs on thousands of chips, such hidden errors can damage a model before anyone notices.
stragglerOne slow computer or chip in a large group working in lockstep. Because everyone must wait for the slowest member at each step, a single straggler can slow down an entire AI training job.
subliminal learningThe finding that an AI model can pass hidden traits — a favorite animal, or bad behavior — to another model through training data that seems to have nothing to do with them.
synthetic dataData created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
temperatureA setting that controls how adventurous an AI model's word choices are. Low temperature makes it pick the safest, most likely words; high temperature makes it more varied and surprising, but more error-prone.
textual inversionTeaching an image generator a new 'word' for a specific object or style from a few example pictures, leaving the model itself unchanged — only one vocabulary entry is added.
the bitter lessonThe argument, made by AI pioneer Rich Sutton in 2019, that AI has advanced mostly through general methods that use ever more computing power, not by programming in human knowledge.
Thinking Machines LabMira Murati's AI company, which she started in San Francisco in 2025 after serving as OpenAI's technology chief. It makes tools for customizing AI models and releases some models openly.
tool useAn AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work.
ToolformerA 2023 Meta research model that learned on its own when a tool — a calendar, a translator, a search engine, a calculator — would help it predict text, keeping only the tool uses that actually paid off.
TPUGoogle's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud.
trainingThe process by which an AI model learns: it is shown huge numbers of examples and its internal settings are adjusted, bit by bit, until its answers get better.
training gamingWhen an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.
training stepOne round of learning: the model makes predictions on a small batch of examples, measures its error and adjusts its settings once. Training runs consist of thousands to millions of steps.
trajectoryThe record of what happened as an AI agent acted: the sequence of situations it was in, the actions it took and the rewards it received.
transfer learningReusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data.
value learningTeaching an AI what people want by having it learn from their behavior, choices and feedback, rather than trying to write every goal down in advance.
vanishing gradient problemA training difficulty in deep or recurrent networks: the learning signal shrinks as it travels back through many layers or time steps, so the early parts of the network barely learn.
wafer-scale engineA single giant processor made from nearly a whole dinner-plate-sized silicon wafer instead of the small chips normally cut from it, so data rarely has to leave the chip.
wav2vecA family of Meta speech models that learn the sounds of language from thousands of hours of unlabeled recordings, so that only minutes of transcribed speech are needed to build a recognizer.
weak-to-strong generalizationThe finding that a strong AI trained only on answers from a weaker, error-prone teacher can end up outperforming that teacher — studied as a stand-in for humans supervising smarter-than-human AI.
weight decayA training rule that shrinks every weight slightly at each step, like a small tax on large numbers. It keeps models simpler and usually helps them work better on new data.
weight initializationChoosing the random starting values of a neural network's internal numbers before training. Good starting values keep signals from fading away or blowing up as they pass through many layers.
win rateThe share of head-to-head comparisons in which judges — people or another AI — prefer one model's answer over a rival's for the same request.