NVLink
In plain wordsNvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip.
Nvidia's proprietary interconnectinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one. for linking GPUsGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time. to one another and to Nvidia CPUs, with far higher bandwidth than PCI Express and support for direct reads and writes into another GPU's memory. Per-GPU bandwidth has roughly doubled each generation: 160 GB/s (NVLink 1, Pascal, 2016), 300 GB/s (Volta), 600 GB/s (Ampere), 900 GB/s (HopperH100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power.), 1.8 TB/s (Blackwell) and 3.6 TB/s (Rubin). NVSwitch chips let every GPU in a domain communicate with every other at full speed; in rack-scale systemsrack-scale system, n.: An AI computer designed as one whole cabinet of chips, cooling and wiring, rather than separate servers, so all the chips inside can work together like one big chip. such as the GB200 NVL72, 72 GPUs share one NVLink domain with 130 TB/s of aggregate bandwidth. This scale-upscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out). domain carries the bandwidth-hungry traffic of tensor parallelismtensor parallelism, n.: Splitting the individual giant tables of numbers inside each layer of a model across several chips, so that every chip does a slice of every calculation at the same time. and expert parallelismexpert parallelism, n.: A way of running a 'mixture of experts' model — one made of many specialist sub-networks — by giving each chip its own set of specialists and sending each piece of text to whichever chip holds the specialists it needs. in large models. NVLink Fusion (2025) licenses the technology to other chip designers.
Origin
Announced by Nvidia in March 2014 and first shipped with the Pascal-generation Tesla P100 (2016), to overcome the PCI Express bottleneck between GPUs and to CPUs such as IBM's POWER processors.
Usage note
Open alternatives include UALink, backed by AMD, Intel, Google and others. The NVLink-C2C variant joins CPU and GPU dies inside Nvidia's 'superchips'.
Related
- interconnectinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one.
- scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).
- rack-scale systemrack-scale system, n.: An AI computer designed as one whole cabinet of chips, cooling and wiring, rather than separate servers, so all the chips inside can work together like one big chip.
- InfiniBandInfiniBand, prop. n.: A very fast networking technology used to connect the thousands of servers in supercomputers and AI training clusters, designed so data moves with minimal delay.
- tensor parallelismtensor parallelism, n.: Splitting the individual giant tables of numbers inside each layer of a model across several chips, so that every chip does a slice of every calculation at the same time.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
- InfiniBandInfiniBand, prop. n.: A very fast networking technology used to connect the thousands of servers in supercomputers and AI training clusters, designed so data moves with minimal delay.
Mentioned in
GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time., NvidiaNvidia, prop. n.: An American chip company whose graphics processors became the main hardware for training and running AI, making it one of the most valuable companies in the world., distributed trainingdistributed training, n.: Training one AI model on many computer chips at once, splitting up the data or the model itself among them and keeping them in sync, so that jobs that would take a single chip thousands of years finish in weeks., H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power., interconnectinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one., 3D parallelism3D parallelism, n.: Combining three ways of dividing up training work — by data, by layer stage and by slices within layers — so that thousands of chips can train one giant model efficiently., disaggregated servingdisaggregated serving, n.: Running the two stages of an AI's reply, reading the prompt and writing the answer, on separate groups of chips, each tuned for its own job, and handing the work between them., InfiniBandInfiniBand, prop. n.: A very fast networking technology used to connect the thousands of servers in supercomputers and AI training clusters, designed so data moves with minimal delay., NCCLNCCL, prop. n.: NVIDIA's software library that lets its graphics chips exchange data with each other quickly — inside one computer and across networks — and that nearly all large-scale AI training on NVIDIA hardware relies on., rack-scale systemrack-scale system, n.: An AI computer designed as one whole cabinet of chips, cooling and wiring, rather than separate servers, so all the chips inside can work together like one big chip., scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out)., tensor parallelismtensor parallelism, n.: Splitting the individual giant tables of numbers inside each layer of a model across several chips, so that every chip does a slice of every calculation at the same time.