Skip to content
Last Dictionary
Plane
Hardware & ComputeHistory & LandmarksDeep Learning

GPGPU

abbreviationAlso: general-purpose computing on graphics processing units, GPU computing, accelerated computing

In plain wordsUsing a graphics chip for ordinary number-crunching instead of drawing pictures — the trick that, years later, made modern AI training practical.

  1. The practice of running non-graphics computations — physics simulation, linear algebra and later neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. training — on a GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.. Early GPGPU (c. 2001–2006) disguised data as textures and computations as pixel shaders written through graphics interfaces such as OpenGL and DirectX, while research languages such as Stanford's Brook offered stream-programming abstractions. Dedicated compute platforms — CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI. (2006–2007), OpenCL (2008–2009) and later ROCmROCm, prop. n.: AMD's free, open-source software for programming its graphics and AI chips — its answer to Nvidia's CUDA, letting AI software run on AMD hardware. — exposed the GPU directly as a parallel processor, letting researchers write matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing. and convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location. code for it and setting the stage for AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.. 'GPU computing' and 'accelerated computing' have largely replaced the acronym.

Origin

Mark Harris, then a graduate student at the University of North Carolina, coined 'GPGPU' in 2002 and founded the community site GPGPU.org. Neural networks were run on GPUs by Kyoung-Su Oh and Keechul Jung (2004) and by Kumar Chellapilla, Sidd Puri and Patrice Simard (2006), before CUDA made the approach mainstream.

Related

See also
  • GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
  • CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI.
  • ROCmROCm, prop. n.: AMD's free, open-source software for programming its graphics and AI chips — its answer to Nvidia's CUDA, letting AI software run on AMD hardware.
  • AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.
  • deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems.
  • hardware lotteryhardware lottery, n.: The idea that research ideas succeed partly by luck — because they happen to suit the computer chips available at the time — rather than purely because they are better.

Sources

  1. Oh, K.-S. & Jung, K. (2004). GPU implementation of neural networks. Pattern Recognition 37(6).
  2. Chellapilla, K., Puri, S. & Simard, P. (2006). High Performance Convolutional Neural Networks for Document Processing. Tenth International Workshop on Frontiers in Handwriting Recognition.

Mentioned in

GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time., CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI.