compute
In plain wordsThe raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
Computational resources treated as a measurable quantity, used for trainingtraining, n.: The process by which an AI model learns: it is shown huge numbers of examples and its internal settings are adjusted, bit by bit, until its answers get better. or running AI modelsAI model, n.: The trained core of an AI: a large file of learned numbers plus the recipe for using them, which turns an input such as a question or a photo into an output.. Narrowly, compute is the number of arithmetic operations performed, counted in FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.; loosely, it is the hardware that supplies them — GPUsGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time., TPUsTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud. and other acceleratorsAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models., with their memory, interconnectsinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one. and power — counted in chips, GPU-hoursGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour. or megawatts. Training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules. for frontier models has grown far faster than Moore's lawMoore's law, n.: Gordon Moore's prediction that chipmakers could pack twice as many transistors onto a chip roughly every two years — a pace that for decades made computers steadily faster and cheaper, and has slowed but not stopped.: OpenAI estimated a doubling every 3.4 months for the largest runs between 2012 and 2018. Scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments. tie model quality to compute, which makes it central to forecasting, business strategy and compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI., since it is measurable, physically concentrated and dependent on a narrow semiconductor supply chain. Spending more compute at inference time (test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter.) has become a second axis of scaling.
We're compute-constrained until the new cluster comes online in the spring.
Origin
The noun 'compute' for computing resources was uncommon before the 2010s and spread through cloud computing ('compute instances'). In AI it became a term of art after Dario Amodei and Danny Hernandez's OpenAI post 'AI and Compute' (May 2018), which charted the training compute of landmark systems; it later entered policy, as in the compute thresholds of a 2023 US executive order and the EU AI Act.
Usage note
A mass noun ('more compute', not 'more computes'). Distinguish the hardware sense from training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules. (operations used by one run) and effective computeeffective compute, n.: A way of measuring AI progress that counts better methods as if they were extra computing power: if a new technique lets a model match one trained with twice the computing, it counts as doubling effective compute., which also credits algorithmic improvements.
Related
- FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
- AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models.
- training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI.
- GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour.
- AI data centerAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city.
- effective computeeffective compute, n.: A way of measuring AI progress that counts better methods as if they were extra computing power: if a new technique lets a model match one trained with twice the computing, it counts as doubling effective compute.
Sources
- Amodei, D. & Hernandez, D. (2018). AI and Compute. OpenAI blog. openai.com/index/ai-and-compute
Mentioned in
AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models., AI bubbleAI bubble, n.: The worry that investors are pouring far more money into AI companies, chips and data centers than AI will earn back soon, so that prices could crash as in the dot-com bust of 2000., AI data centerAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city., environmental impact of AIenvironmental impact of AI, n.: The effects of AI on the planet: the electricity, water, materials and land used to make chips, build data centers and train and run models, and the pollution that results., FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules., AI stackAI stack, n.: The layers of technology an AI product depends on, from chips and data centers at the bottom, through the models, up to the apps people use, each built on the layer below., AI supercomputerAI supercomputer, n.: A huge network of AI chips — often tens or hundreds of thousands — wired together in a data center to train the largest AI models., AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never., compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI., concentration of powerconcentration of power, n.: The worry that AI will put enormous economic, political or military power in the hands of a few companies, governments or individuals, beyond the reach of ordinary checks and balances., Epoch AIEpoch AI, prop. n.: A research nonprofit that gathers data on how fast AI is advancing — the computing power, data and money behind it — and publishes forecasts and difficult tests for AI., GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour., H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power., hyperscalerhyperscaler, n.: One of the few giant technology companies — such as Amazon, Microsoft, Google and Meta — that run vast networks of data centers and spend most of the money that builds AI computing., intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time., Jevons paradoxJevons paradox, n.: The observation that making something more efficient can make people use more of it overall because it gets cheaper — as with coal in the 1800s, and with AI computing today., moatmoat, n.: A lasting business advantage that keeps competitors out, like a castle's moat; in AI, the question of whether any company can stay ahead when rivals and free models catch up quickly., neocloudneocloud, n.: A newer kind of cloud company that mainly rents out powerful AI chips, rather than the full range of computing services that giants like Amazon or Microsoft sell., the bitter lessonthe bitter lesson, phr.: The argument, made by AI pioneer Rich Sutton in 2019, that AI has advanced mostly through general methods that use ever more computing power, not by programming in human knowledge., training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules., CERN for AICERN for AI, n.: A proposed publicly funded, international AI laboratory modeled on CERN, the European particle-physics center, where countries would pool money, computers and scientists to do AI research together., circular dealcircular deal, n.: A business arrangement in which an AI chip or cloud supplier invests in, or lends to, a customer that then spends the money buying the supplier's own products., compute overhangcompute overhang, n.: A situation in which far more computing power is available than AI currently uses, so that a new method or a decision to scale up could suddenly produce a big jump in ability., CoreWeaveCoreWeave, prop. n.: An American company that rents out huge numbers of Nvidia graphics processors over the internet to AI developers such as Microsoft, OpenAI and Meta.