GPU-hour
In plain wordsOne graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour.
A unit of accelerator time: one GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time. (or other accelerator) used for one hour, regardless of how fully its arithmetic units were busy. It is the common currency for reporting and pricing compute. Model reports give training in GPU-hours — Meta's Llama 2 70B used about 1.7 million A100 GPU-hours, and DeepSeek reported 2.788 million H800 GPU-hours for DeepSeek-V3, which at an assumed $2 per GPU-hour implied a training cost of about $5.6 million — and cloud providers rent accelerators by the GPU-hour. Comparisons across generations need care, since an H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power. hour delivers roughly three times the 16-bit arithmetic of an A100 hour; converting to FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules. requires the chip's peak FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips. and the achieved utilizationmodel FLOPs utilization, n.: The share of a computer's top calculating speed that actually goes into useful AI training work. Real systems often reach only around a third to a half..
The ablation sweep burned 40,000 GPU-hours before we found the bug in the data loader.
Origin
Modeled on CPU-hours and core-hours, long used to allocate supercomputer time; 'GPU-hours' became standard in machine-learning papers in the late 2010s and entered public debate about training costs with reports such as DeepSeek-V3's (December 2024).
Usage note
Headline GPU-hour costs usually cover only the final training run, excluding experiments, failed runs, data, staff and hardware ownership.
Related
- computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
- training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules.
- inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text.
- petaflop/s-daypetaflop/s-day, n.: A unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.
- H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power.
Sources
- Touvron, H. et al. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288.
- DeepSeek-AI (2024). DeepSeek-V3 Technical Report. arXiv:2412.19437.
Mentioned in
computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress., compute budgetcompute budget, n.: The total amount of computing a team can spend on training (or running) a model, set in advance like a money budget and used to decide how big the model should be and how much data it sees., H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power., petaflop/s-daypetaflop/s-day, n.: A unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.