TOPS
In plain wordsTrillions of operations per second: the speed rating usually given for AI chips in phones and laptops, often counting simple whole-number math rather than precise fractions.
A throughput rating of 10^12 operations per second, used mainly for NPUsNPU, abbr.: A small AI chip, or part of a chip, built into phones, laptops and other devices so they can run AI features quickly without draining the battery or using the cloud. and inference accelerators whose arithmetic is integer rather than floating-point. By convention a multiply–accumulate counts as two operations, and unless stated otherwise TOPS figures assume INT8INT8, abbr.: Storing numbers as whole numbers from −128 to 127 using 8 bits. AI models converted to this format run faster and need a quarter of the memory of 32-bit versions. arithmetic at peak clock, sometimes doubled by assuming structured sparsitysparsity, n.: The property of being mostly zeros or mostly switched off. AI systems exploit it by skipping the empty parts, so they need less memory and computation.. Energy efficiency is reported in TOPS per watt. Because the rating ignores memory bandwidth, utilization and precision, two chips with equal TOPS can differ widely in real model speed. Microsoft's 2024 requirement of at least 40 NPU TOPS for Copilot+ PCs turned the figure into a consumer marketing number.
Origin
Used for digital signal processors and early neural-network chips before the deep-learning era; Google's first TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud. paper (2017) reported 92 TOPS of 8-bit throughput, and phone and PC makers adopted TOPS ratings for NPUs from 2017 onward.
Related
- FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.
- NPUNPU, abbr.: A small AI chip, or part of a chip, built into phones, laptops and other devices so they can run AI features quickly without draining the battery or using the cloud.
- INT8INT8, abbr.: Storing numbers as whole numbers from −128 to 127 using 8 bits. AI models converted to this format run faster and need a quarter of the memory of 32-bit versions.
- on-device AIon-device AI, n.: AI that runs directly on your own phone, laptop, car or headset rather than on a company's distant servers, so it can work offline, respond instantly and keep your data on the device.
- AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models.
- FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips.
Mentioned in
FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips., INT8INT8, abbr.: Storing numbers as whole numbers from −128 to 127 using 8 bits. AI models converted to this format run faster and need a quarter of the memory of 32-bit versions., NPUNPU, abbr.: A small AI chip, or part of a chip, built into phones, laptops and other devices so they can run AI features quickly without draining the battery or using the cloud.