FLOP
In plain wordsA single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.
A single arithmetic operation — an addition, multiplication or similar — on floating-pointfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster. numbers, used as a unit of computational work. AI practice measures a model's training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules. in total FLOP: for a dense transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it., roughly 6 FLOP per parameterparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. per training token (about 2 in the forward passforward pass, n.: The step where a neural network takes an input and runs it through its layers, one after another, to produce an output — its answer or prediction. and 4 in the backward passbackward pass, n.: The step in training where the network works backward from its error, layer by layer, to calculate how each internal setting should change.), and about 2 FLOP per parameter per token at inference. Frontier models of 2024–2026 are estimated to have used on the order of 10^25 to 10^26 FLOP or more. Because such counts are easy to estimate and track capability, they anchor scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments. and regulation: the EU AI Act presumes a general-purpose model has 'high-impact capabilities' above 10^25 FLOP of cumulative training compute, and the now-revoked US Executive Order 14110Executive Order 14110, prop. n.: A sweeping 2023 order by US President Joe Biden that told federal agencies how to manage AI risks and required makers of the largest AI models to report their safety tests to the government. It was revoked in 2025. used 10^26 operations as a reporting threshold. A fused multiply–add is conventionally counted as 2 FLOP.
At 6ND, a 70-billion-parameter model trained on 15 trillion tokens takes about 6 × 10^24 FLOP.
Origin
Counting floating-point operations to measure numerical work is long-standing practice in numerical analysis and supercomputing (see FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.). The 6ND approximation was popularized by Jared Kaplan and colleagues' 2020 scaling-laws paper, and FLOP counts entered law with US Executive Order 14110 (October 2023) and the EU AI Act (2024).
Usage note
Usage is inconsistent: 'FLOP' or 'FLOPs' for a count, 'FLOPS' or 'FLOP/s' for a rate. Writing 'FLOP/s' for rates avoids confusing 'FLOPs' with 'FLOPS'. Multiples take SI prefixes (1 teraFLOP = 10^12 FLOP).
Related
- FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.
- training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules.
- compute thresholdcompute threshold, n.: A legal cutoff based on how much computation was used to train an AI model. Models trained with more than that amount face extra rules, on the theory that bigger training runs produce more capable models.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- petaflop/s-daypetaflop/s-day, n.: A unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.
- model FLOPs utilizationmodel FLOPs utilization, n.: The share of a computer's top calculating speed that actually goes into useful AI training work. Real systems often reach only around a third to a half.
- computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
- FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.
Sources
- Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
Defined in law or standards
EU AI Act (Reg. (EU) 2024/1689), Art. 3(67) ('floating-point operation'); Art. 51(2) (presumption of high-impact capabilities above 10^25 FLOP)
Mentioned in
computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress., frontier AIfrontier AI, n.: The most capable general-purpose AI systems of the moment — those at or beyond the leading edge — which governments single out for extra safety attention., general-purpose AI modelgeneral-purpose AI model, n.: The EU's legal name for the large, adaptable AI models behind chatbots and similar tools: models that can turn their hand to many different jobs and be built into many other products., compute budgetcompute budget, n.: The total amount of computing a team can spend on training (or running) a model, set in advance like a money budget and used to decide how big the model should be and how much data it sees., compute thresholdcompute threshold, n.: A legal cutoff based on how much computation was used to train an AI model. Models trained with more than that amount face extra rules, on the theory that bigger training runs produce more capable models., FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips., general-purpose AI model with systemic riskgeneral-purpose AI model with systemic risk, n.: The EU's label for the very largest and most capable AI models, whose failure or misuse could cause harm across society, so their makers must test them for dangers and report serious incidents., GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour., matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing., parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them., training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules., arithmetic intensityarithmetic intensity, n.: How much calculating a task does for each piece of data it fetches from memory. Tasks with little calculation per fetch are slowed by memory, not by the chip's math speed., IsoFLOP analysisIsoFLOP analysis, n.: A way of finding the best model size for a given amount of computing: train several models of different sizes at exactly the same computing cost and see which one ends up best., model FLOPs utilizationmodel FLOPs utilization, n.: The share of a computer's top calculating speed that actually goes into useful AI training work. Real systems often reach only around a third to a half., petaflop/s-daypetaflop/s-day, n.: A unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.