computer vision
In plain wordsThe part of AI that lets computers make sense of images and video: recognizing faces, reading signs, spotting tumors on scans or judging how far away a car is.
The field of AI and engineering concerned with extracting useful information from images, video and other visual signals: recognizing what is present (image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities., object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene., image segmentationimage segmentation, n.: Dividing a picture into regions by deciding, pixel by pixel, which object or kind of surface each tiny dot belongs to, like coloring inside the outline of everything.), where things are and how they move (depth estimationdepth estimation, n.: Working out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far., optical flowoptical flow, n.: How each part of a video picture seems to shift from one frame to the next, recorded as a tiny arrow per pixel giving the direction and distance moved., object trackingobject tracking, n.: Following objects from frame to frame in a video and keeping track of which is which, so a system knows that the car it sees now is the same car it saw a second ago.), and the three-dimensional structure behind the pixels (3D reconstruction3D reconstruction, n.: Building a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans.). Early systems combined hand-designed operators, such as edge and corner detectors and SIFTscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles. descriptors, with geometry and statistical classifiers. Since AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward. won the 2012 ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom. challenge, the field has been dominated by learned representations: first convolutional networksconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., then vision transformersvision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence. and models pretrained with self-supervisionself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image. or on image–text pairs (CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image.). By the mid-2020s much perception was handled by general foundation modelsfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools., including vision-language modelsvision-language model, n.: An AI model that understands images and text together — it can describe a photo, answer questions about a chart or find the picture that matches a sentence. that answer questions about images in natural language. Applications include medical imaging, driver assistance, industrial inspection, photography, retail, satellite analysis and robotics.
Our computer vision pipeline flags damaged parcels on the conveyor before they reach the loading dock.
Origin
Lawrence Roberts's 1963 MIT thesis on recovering three-dimensional block shapes from photographs is often called the field's founding work. MIT's 1966 Summer Vision Project, proposed by Seymour Papert, hoped to build 'a significant part of a visual system' with summer students. The phrase 'computer vision' was established by the mid-1970s, appearing in the title of Patrick Winston's edited volume The Psychology of Computer Vision (1975) and later of Dana Ballard and Christopher Brown's textbook (1982); its earliest use has not been traced.
Usage note
'Machine vision' often denotes industrial inspection and guidance systems built around cameras and controlled lighting; 'image processing' transforms images (denoising, compression) without necessarily interpreting them. Computer vision is often described as the inverse of computer graphics: graphics renders images from descriptions of scenes, vision infers scenes from images.
Related
- perceptionperception, n.: How an AI system or robot turns raw sensor signals, such as camera pixels, microphone sound or laser scans, into an understanding of what is around it: objects, people, words and distances.
- image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities.
- object detectionobject detection, n.: Finding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene.
- image segmentationimage segmentation, n.: Dividing a picture into regions by deciding, pixel by pixel, which object or kind of surface each tiny dot belongs to, like coloring inside the outline of everything.
- vision transformervision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence.
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- vision-language modelvision-language model, n.: An AI model that understands images and text together — it can describe a photo, answer questions about a chart or find the picture that matches a sentence.
- pattern recognitionpattern recognition, n.: The ability of a computer, or a brain, to spot regularities in data and sort things into categories, such as recognizing handwritten letters, faces or spoken words.
Sources
- Roberts, L. G. (1963). Machine Perception of Three-Dimensional Solids. PhD thesis, Massachusetts Institute of Technology.
- Papert, S. (1966). The Summer Vision Project. MIT Artificial Intelligence Group, Vision Memo No. 100 (AIM-100).
- Szeliski, R. (2022). Computer Vision: Algorithms and Applications, 2nd ed. Springer.
Mentioned in
artificial intelligenceartificial intelligence, n.: Computer systems that do things we would normally credit to a thinking mind, such as holding a conversation, reading an X-ray or planning a route, and the field of research that builds them., dronedrone, n.: An aircraft that carries no one aboard to fly it: a person steers it from the ground, or its own software does. Small camera-carrying quadcopters are the most familiar kind., facial recognitionfacial recognition, n.: Technology that identifies or checks who someone is by comparing a picture of their face with stored photos, used by phones, passport gates and police databases., lethal autonomous weapons systemlethal autonomous weapons system, n.: A weapon that, after a person sets it going, picks out and attacks targets — including people — on its own, with no human deciding each strike. Critics call them killer robots., AI-completeAI-complete, adj.: Describes a problem believed to be so hard that solving it would require general, human-level intelligence, such as flawless translation or truly understanding a story., convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., CVPRCVPR, prop. n.: The leading yearly conference on computer vision — teaching computers to understand images and video — and one of the most cited publication venues in any field of science., Fei-Fei LiFei-Fei Li, prop. n.: A Chinese-American computer scientist whose ImageNet project, millions of hand-labeled photos, gave deep learning the proving ground that launched its boom; she now champions 'human-centered' AI., Kaiming HeKaiming He, prop. n.: A Chinese computer scientist whose ResNet design, with shortcut connections that let networks grow hundreds of layers deep, is among the most cited research in all of science., pattern recognitionpattern recognition, n.: The ability of a computer, or a brain, to spot regularities in data and sort things into categories, such as recognizing handwritten letters, faces or spoken words., perceptionperception, n.: How an AI system or robot turns raw sensor signals, such as camera pixels, microphone sound or laser scans, into an understanding of what is around it: objects, people, words and distances., Alex KrizhevskyAlex Krizhevsky, prop. n.: A Canadian computer scientist who built AlexNet, the image-recognition network that was trained on gaming graphics chips and whose 2012 contest win launched the deep learning boom., blocks worldblocks world, n.: A simplified pretend world of toy blocks on a table that early AI researchers used to test programs for planning, seeing and understanding language, because it was simple enough to describe completely., ICCVICCV, prop. n.: A major international computer vision conference, held in odd-numbered years, where much of the research on how machines interpret images and video is first presented., neocognitronneocognitron, n.: A 1980 neural network for recognizing handwritten characters, modeled on the brain's visual system. Its stacked layers of pattern detectors were the blueprint for the networks used in modern image recognition., optical flowoptical flow, n.: How each part of a video picture seems to shift from one frame to the next, recorded as a tiny arrow per pixel giving the direction and distance moved., scale-invariant feature transformscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles., Seymour PapertSeymour Papert, prop. n.: A South African-born mathematician who co-wrote 'Perceptrons', a critique of early neural networks, with Marvin Minsky, and invented the children's programming language Logo., Strategic Computing InitiativeStrategic Computing Initiative, prop. n.: A billion-dollar US defense research program, run from 1983 to 1993, that tried to build machine intelligence for military uses such as self-driving vehicles and assistants for fighter pilots.