systolic array
In plain wordsA grid of tiny calculators that pass numbers to their neighbors in rhythm, like a heartbeat, so each value is reused many times. AI chips use it to multiply large tables of numbers efficiently.
A regular grid of identical processing elements, each connected only to its neighbors, through which data are pumped in lockstep: every clock cycle each cell takes inputs from adjacent cells, performs a multiply–accumulate and passes values on. Because each operand fetched from memory is reused across an entire row or column of cells, a systolic array achieves very high arithmetic intensityarithmetic intensity, n.: How much calculating a task does for each piece of data it fetches from memory. Tasks with little calculation per fetch are slowed by memory, not by the chip's math speed. with short, local wiring, which suits matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing. and convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.. In a weight-stationary design, as in Google's first TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud. (a 256 × 256 array), weights stay in the cells while activations flow through and partial sums accumulate along the array. Many AI acceleratorsAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models. and NPUsNPU, abbr.: A small AI chip, or part of a chip, built into phones, laptops and other devices so they can run AI features quickly without draining the battery or using the cloud. use systolic or similar dataflow arrays for their matrix units.
Origin
Introduced by H. T. Kung and Charles E. Leiserson in 'Systolic Arrays (for VLSI)' (presented 1978, published 1979) and popularized by Kung's 1982 IEEE Computer article 'Why Systolic Architectures?'. The name likens the rhythmic data flow to the heart's systole, when it pumps blood.
Related
- TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud.
- tensor coretensor core, n.: A small, specialized part of an AI chip that does one job — multiplying small blocks of numbers together — far faster than ordinary circuits can.
- matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing.
- AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models.
- arithmetic intensityarithmetic intensity, n.: How much calculating a task does for each piece of data it fetches from memory. Tasks with little calculation per fetch are slowed by memory, not by the chip's math speed.
Sources
- Kung, H. T. & Leiserson, C. E. (1979). Systolic Arrays (for VLSI). In Sparse Matrix Proceedings 1978. SIAM.
- Kung, H. T. (1982). Why Systolic Architectures? IEEE Computer 15(1).
Mentioned in
AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models., architecturearchitecture, n.: The overall design of an AI model, meaning what kinds of parts it has and how they are connected, as opposed to the specific numbers it learns during training., matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing., tensor coretensor core, n.: A small, specialized part of an AI chip that does one job — multiplying small blocks of numbers together — far faster than ordinary circuits can., TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud., FPGAFPGA, abbr.: A chip whose circuits can be rewired by software after it is made, so engineers can turn it into a custom processor — slower than a purpose-built chip, but flexible.