Vision, Speech & Perception

Seeing and hearing: computer vision, speech and audio. 68 entries live here, and 140 from other domains touch it.

Essential

computer visionThe part of AI that lets computers make sense of images and video: recognizing faces, reading signs, spotting tumors on scans or judging how far away a car is.
image classificationTeaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities.
object detectionFinding every object of interest in a picture or video frame, saying what each one is and drawing a box around it, for example all the cars, cyclists and pedestrians in a street scene.
optical character recognitionSoftware that picks out the letters and words in a picture — a road sign, a handwritten note, a page of an old book — and turns them into real text a computer can copy, search or read aloud.
speech recognitionTechnology that turns spoken words into written text, as when a phone transcribes a voice message or captions appear automatically under a video.
text-to-speechTechnology that reads written text aloud in a computer-generated voice, used in screen readers, voice assistants, audiobooks and navigation apps.

All other entries

3D reconstructionBuilding a 3D model of an object or place, its shape and often its colors, from photos, video or depth-sensor scans.
action recognitionTeaching computers to recognize what people are doing in a video, such as running, waving, cooking or falling, rather than just which objects appear in it.
affective computingTechnology that tries to read people's emotions — from their faces, voices or words — or to display emotions of its own, like software that guesses your mood or a robot that seems to show feelings.
anchor boxA ready-made box shape, placed at many spots across an image, that an object detector stretches and shifts to fit real objects instead of drawing boxes from scratch.
audio language modelAn AI language model that can take in sound, such as speech, music and everyday noises, and understand or generate it, much as text chatbots handle words.
backboneThe main, usually pretrained part of a vision model that turns an image into useful internal features; smaller task-specific parts are attached to it, like limbs to a spine.
bounding boxThe rectangle a vision system draws around an object it has found, stored as a handful of numbers giving the rectangle's position and size.
CLIPAn OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image.
COCOA widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems.
ConformerA neural-network design for speech recognition that combines a transformer, which relates distant parts of a recording, with convolutions, which pick up fine local sound patterns.
depth estimationWorking out how far away each point in a scene is from the camera, producing a 'depth map' that turns a flat picture into a sense of near and far.
DETRAn object detector built on the transformer, the design behind modern chatbots, that outputs a short list of objects in one go instead of sorting through thousands of guessed boxes.
DINOA family of Meta vision models that learn to understand images from millions of unlabeled photos, by training one network to agree with a slowly updated copy of itself.
document understandingUsing AI to read documents such as invoices, forms, contracts and scientific papers the way a person does, taking in layout, tables and charts as well as words, to pull out the information needed.
face detectionFinding where the faces are in a photo or video, without working out whose faces they are: the step that lets a camera focus on people or blur them out.
feature extractionTurning raw data such as pixels or sound into a short list of telling numbers — computed by hand-designed rules or by an AI system trained earlier — that a new model can learn from or compare.
Gaussian splattingA way of turning photos of a real place into a 3D scene made of millions of tiny, soft, colored blobs that can be viewed from any angle in real time, even on a phone or headset.
image captioningHaving a computer write a short description of a picture, such as 'a dog catching a frisbee in a park', to help blind people, search engines and other AI systems.
image segmentationDividing a picture into regions by deciding, pixel by pixel, which object or kind of surface each tiny dot belongs to, like coloring inside the outline of everything.
instance segmentationFinding each separate object in an image and tracing its exact outline, so that two people standing together get two distinct shapes rather than one blob.
intersection over unionA score from 0 to 1 for how well two shapes overlap, such as a predicted box and the true box around an object: the shared area divided by the combined area.
joint-embedding predictive architectureA way of training AI on images or video in which the model predicts an abstract summary of a hidden part, rather than every pixel, so it learns what matters and ignores unpredictable detail.
keypoint detectionFinding specific, recognizable points in an image, such as the corners of a building, a person's elbows and knees, or the tip of a nose.
LibriSpeechA free collection of about 1,000 hours of volunteers reading English audiobooks, matched with the book text; it became the standard yardstick for speech-recognition systems.
masked autoencoderA way to teach a vision model without labels: hide most of an image's tiles and train the model to fill in the missing parts, so that it learns what images are made of.
mean average precisionThe standard score for object detectors: it rewards finding all the objects while raising few false alarms, averaged over every type of object. Higher is better, up to 100.
mean opinion scoreA quality score from 1 (bad) to 5 (excellent) obtained by asking many listeners to rate audio, such as a synthetic voice or a phone call, and averaging their ratings.
mel spectrogramA version of a sound picture whose pitch axis is squeezed to match human hearing, giving more detail to low sounds than to high ones; the usual way AI models take in audio.
MFCCA short list of numbers, usually about 13, that summarizes the shape of a brief slice of sound in a way that mimics human hearing; for decades the standard input to speech recognizers.
neural radiance fieldA way of capturing a real scene from a set of photos as a small neural network that can then show the scene from any new viewpoint, as if moving a camera through it.
non-maximum suppressionA clean-up step in object detection: when the system draws several overlapping boxes around the same object, it keeps the most confident one and throws away the near-duplicates.
object trackingFollowing objects from frame to frame in a video and keeping track of which is which, so a system knows that the car it sees now is the same car it saw a second ago.
open-vocabulary detectionObject detection where you type what to look for, such as 'red umbrella' or 'forklift', and the system finds it, even if it was never trained on that exact category.
optical flowHow each part of a video picture seems to shift from one frame to the next, recorded as a tiny arrow per pixel giving the direction and distance moved.
panoptic segmentationA complete labeling of an image in which every pixel gets a category, such as road or sky, and every countable object, such as each car or person, also gets its own identity.
patch embeddingThe first step of a vision transformer: cutting an image into small square tiles and turning each tile into a list of numbers that the model can process like a word.
perceptionHow an AI system or robot turns raw sensor signals, such as camera pixels, microphone sound or laser scans, into an understanding of what is around it: objects, people, words and distances.
point cloudA 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos.
pose estimationWorking out the position and orientation of something from images: where a person's joints are and how their body is posed, or exactly how an object or camera is placed and turned in space.
R-CNNA family of object detectors that first pick out image regions likely to contain objects, then use a neural network to label each region and tighten its box.
scale-invariant feature transformA classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles.
Segment Anything ModelA family of AI models from Meta that trace the exact outline of whatever you point to in a photo or video, whether by clicking on it, drawing a box around it or, in newer versions, naming it.
semantic segmentationColoring every pixel of an image by what it shows, such as road, sky, person or car, without telling one person or car apart from another.
sound event detectionTeaching computers to recognize everyday sounds in a recording, such as a dog barking, glass breaking or a siren, and to say exactly when each one starts and stops.
source separationPulling apart a mixture of sounds into its separate parts, such as isolating one voice in a noisy party or the vocals, drums and bass in a finished song.
spatial intelligenceThe ability to understand, imagine and act in three-dimensional space and the physical world — effortless for people, and treated by many AI researchers as the next big goal after language.
speaker diarizationWorking out who spoke when in a recording, splitting a meeting or phone call into turns labeled Speaker 1, Speaker 2 and so on, without necessarily knowing who the speakers are.
speaker verificationChecking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone.
spectrogramA picture of sound that shows time from left to right, pitch from bottom to top, and loudness as brightness or color, so that speech and music can be seen as patterns.
speech-to-speech modelAn AI model that listens to speech and replies or translates directly in speech, without first writing everything down as text, so conversations feel faster and keep the tone of voice.
structure from motionRecovering both the 3D shape of a scene and the spot from which each photo was taken, using nothing but a collection of overlapping photographs.
vision transformerA way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence.
visual groundingLinking words to the exact part of an image they refer to, for example finding 'the man in the red hat on the left' and pointing to him.
visual question answeringAsking an AI a question about a picture, such as 'How many people are wearing hats?', and having it answer in words.
vocoderA device or program that turns a compact description of a voice back into actual sound; in modern speech AI, the final stage that produces the audio you hear.
voice activity detectionDetecting, moment by moment, whether someone is speaking in an audio stream, so a system knows when to listen, when to stop recording and when it is its turn to talk.
wake wordA word or phrase, such as 'Alexa' or 'Hey Google', that a smart speaker or phone listens for constantly and that switches it into full listening mode.
wav2vecA family of Meta speech models that learn the sounds of language from thousands of hours of unlabeled recordings, so that only minutes of transcribed speech are needed to build a recognizer.
WaveNetA DeepMind model that generates speech and other sounds one tiny audio sample at a time, producing voices far more natural than earlier computer speech.
WhisperA free OpenAI model that writes down what is said in recordings in many languages, and can translate it into English; it learned from hundreds of thousands of hours of online audio.
word error rateThe standard score for speech recognition: the share of words a system gets wrong compared with a correct transcript, counting wrong, missing and extra words. Lower is better.
YOLOA family of fast object-detection models that look at an image once and immediately output boxes and labels for everything they recognize, quick enough for live video.

Also relevant

140

adversarial exampleAn input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
adversarial patchA printed sticker or picture designed so that, when it appears in a camera's view, an AI vision system misreads the scene — seeing a toaster, say, or missing a person.
AI handsThe tendency of early AI image generators to draw hands badly — extra or missing fingers, twisted joints — which became a running joke and a quick way to spot fake images.
AI surveillanceUsing AI to watch, identify, track or profile people on a scale no team of humans could manage, for example by scanning camera feeds or analyzing phone and online data.
AlexNetThe neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.
annotationAdding information to raw data by hand or with software — marking objects in photos, tagging names in sentences, transcribing speech or ranking chatbot answers — so machines can learn from it.
autonomous vehicleA vehicle, usually a car, truck or shuttle, that can drive itself, sensing the road, deciding what to do and steering, braking and accelerating with no human driving.
binding problemThe puzzle of how a brain, or a neural network, keeps track of which features belong together — that the red goes with the apple and the green with the leaf — when they are processed separately.
biometric categorisationUsing AI to sort people into groups — by age, sex, ethnicity and the like — from their faces, voices or other body features. The EU bans using it to guess sensitive traits such as religion or sexuality.
biometric dataInformation about your body or behavior — your face, fingerprints, voice, eyes or the way you walk — measured by technology and used to recognize who you are or to guess things about you.
browser agentAn AI agent that uses a web browser on your behalf — searching, clicking links, filling in forms and reading pages — to complete tasks such as booking, shopping or gathering information.
Joy BuolamwiniA Canadian-American computer scientist and activist whose research showed that face-analysis software works far worse on darker-skinned women, and who founded the Algorithmic Justice League.
CaffeAn early free deep-learning toolkit from the University of California, Berkeley, popular for image recognition in the mid-2010s, in which networks were described in simple text files rather than code.
capsule networkA network design championed by Geoffrey Hinton in which groups of neurons, called capsules, each describe a part of an object and its position, so the network can check whether the parts fit together.
CAPTCHAA small test on websites, like typing wavy letters or clicking every picture with a traffic light, meant to be easy for people but hard for computer programs, so as to keep automated bots out.
Clearview AIA US company that collected billions of photos from the internet without permission to build a face-search tool for police, the best-known example of controversial facial recognition.
CLIP scoreA score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions.
computer useAn AI's ability to work a computer through its normal screen and controls: it studies pictures of what is displayed and sends mouse and keyboard actions, so it can use ordinary apps that were built for people.
concept bottleneck modelA model made to first predict human-understandable concepts — such as 'bone spur present' on an X-ray — and then decide only from those concepts, so people can inspect and correct them.
connectionist temporal classificationA training method that lets a network learn to transcribe speech or handwriting without being told exactly where each letter or sound begins and ends.
contrastive learningA way to learn without labels by comparison: the model learns to pull together two versions of the same thing, such as two crops of one photo, and push apart unrelated things.
conversational AIThe technology that lets computers hold conversations with people in everyday language, by text or voice. It powers chatbots, voice assistants and automated customer-service lines.
ConvNeXtA 2022 image-recognition network showing that an older-style convolutional design, updated with lessons from transformers, can match vision transformers.
convolutionAn operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
convolutional neural networkA kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
CVPRThe leading yearly conference on computer vision — teaching computers to understand images and video — and one of the most cited publication venues in any field of science.
data augmentationCreating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly.
deepfake detectionUsing software, often itself AI, to tell whether a video, photo or voice recording has been faked or generated by AI rather than captured from reality.
DenseNetAn image-recognition network in which every layer receives the outputs of all the layers before it, so features are reused rather than relearned.
depthwise separable convolutionA cheaper way to do the main operation of image networks: first scan each channel of the input separately, then mix the channels together with a tiny one-by-one step.
dilated convolutionA version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost.
EfficientNetA family of image-recognition networks from 2019 that grows a small base design in a balanced way, deeper, wider and sharper together, to get more accuracy for each unit of computing.
emotion recognition systemAI that tries to work out how people feel or what they intend from their faces, voices or bodies. The EU bans it at work and in schools and tightly regulates it elsewhere.
end-to-end learningTraining one AI system to go straight from raw input, such as camera images, to the final output, such as steering commands, instead of chaining hand-designed steps.
facial recognitionTechnology that identifies or checks who someone is by comparing a picture of their face with stored photos, used by phones, passport gates and police databases.
feature mapThe grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
feature visualizationMaking pictures that show what a part of an image-recognition network is looking for, by repeatedly adjusting an image until that part responds as strongly as possible.
filterIn an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture.
focal lossA way of scoring a model's mistakes during training that pays less attention to examples it already gets right easily, so that learning concentrates on the hard and rare cases.
Fourier transformA way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors.
frameIn classic AI, a ready-made template for a familiar kind of thing or event, such as a kitchen or a job interview, with blanks to fill in and sensible default values.
Kunihiko FukushimaA Japanese engineer who in 1979 invented the neocognitron, a brain-inspired image-recognition network that was the direct ancestor of today's convolutional neural networks.
fully convolutional networkAn image network made only of scanning-window layers, with no fixed-size final layers, so it can take pictures of any size and label every pixel.
Timnit GebruAn Ethiopian-born American computer scientist who exposed racial and gender bias in AI systems, co-wrote the 'stochastic parrots' paper, and founded an independent AI research institute.
GeminiGoogle's family of AI models and its chatbot assistant, launched in December 2023. Gemini models handle text, images, audio and video and power many Google products.
Gemini RoboticsA family of AI models from Google DeepMind, built on its Gemini models, that let robots understand what they see and what people ask of them, and turn that understanding into physical actions.
Gender ShadesA 2018 study showing that commercial face-analysis software guessed the gender of darker-skinned women far less accurately than that of lighter-skinned men.
Grad-CAMA technique that shows, as a coarse heat map, which regions of an image a vision network relied on when choosing a particular label.
graspingHow a robot picks up and holds an object securely with a gripper or hand: deciding where to put its fingers or suction cup and how hard to squeeze.
groundingConnecting an AI's words to something real — trusted documents, a live search, images or the physical world — so that its answers rest on evidence rather than only on what it absorbed in training.
GUI agentAn AI agent that works through the buttons, menus and screens of ordinary software — on a desktop, phone or website — instead of through behind-the-scenes programming interfaces.
guided backpropagationA way of producing sharp, detailed pictures of which parts of an image a vision network responded to, by passing only positive signals backward through the network.
Kaiming HeA Chinese computer scientist whose ResNet design, with shortcut connections that let networks grow hundreds of layers deep, is among the most cited research in all of science.
hidden Markov modelA model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds.
ICCVA major international computer vision conference, held in odd-numbered years, where much of the research on how machines interpret images and video is first presented.
image tokenizerA component that chops a picture into a short sequence of 'visual words' that a language-style AI model can read or write, and turns such sequences back into pictures.
ImageNetA huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom.
image-to-image translationTurning one picture into another version of itself — a sketch into a photo, a day scene into night, a horse into a zebra — while keeping its layout.
implicit neural representationA way of storing an image, shape or scene as a small neural network that, given any coordinate, returns what is there, such as a color or density, instead of as a grid of pixels.
InceptionA family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low.
independent component analysisA technique for unmixing signals that have been blended together, such as separating individual voices from several microphones recording a crowded room.
inpaintingFilling in a missing or erased part of a picture so it blends seamlessly with the rest — removing a stranger from a photo, or repairing a damaged one.
instruction-based image editingChanging a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand.
invarianceA property of a model whose answer does not change when the input is altered in an irrelevant way, such as a photo classifier that still says 'cat' when the cat moves or the picture is flipped.
Jaccard similarityA simple score for how much two collections overlap: the number of items they share divided by the number of distinct items in either. Identical sets score 1; sets with nothing in common score 0.
Frederick JelinekA Czech-born American engineer who led IBM's speech-recognition research and made statistics learned from data, rather than hand-written grammar rules, the basis of language technology.
Alex KrizhevskyA Canadian computer scientist who built AlexNet, the image-recognition network that was trained on gaming graphics chips and whose 2012 contest win launched the deep learning boom.
Ray KurzweilAn American inventor of reading machines and speech technology who became the best-known prophet of the 'singularity', predicting human-level AI by 2029.
LAION-5BA free list of nearly six billion links to online images paired with their captions, released in 2022 and used to train image generators such as Stable Diffusion.
layer-wise relevance propagationA method that traces a neural network's decision backward, layer by layer, handing each part of the input its share of the credit until every pixel or word has a relevance score.
Yann LeCunA French-American computer scientist who pioneered the neural networks that recognize images, led Meta's AI research for over a decade, and argues that chatbots alone will not reach human-level AI.
LeNetA family of early image-recognition neural networks built at Bell Labs from 1989 that learned to read handwritten digits. A version read huge numbers of bank checks, and its design underlies today's image models.
Fei-Fei LiA Chinese-American computer scientist whose ImageNet project, millions of hand-labeled photos, gave deep learning the proving ground that launched its boom; she now champions 'human-centered' AI.
lidarA sensor that measures distances by firing laser pulses and timing their reflections, building a 3D picture of its surroundings out of millions of points; widely used on robots and self-driving cars.
long short-term memoryA type of recurrent network with a separate memory lane and learned 'gates' that decide what to remember, forget and output, letting it keep track of information over long sequences.
manipulationA robot's use of arms, hands or grippers to change the world around it: picking things up, moving, pushing, opening, assembling or using tools.
Markov random fieldA model of many linked uncertain quantities, such as neighboring pixels in a photo, in which each one depends directly only on its neighbors, with no notion of which one causes which.
Marr's levels of analysisA way of explaining any information-processing system on three levels: what problem it solves and why, what step-by-step method it uses, and how that method is physically built.
Microsoft ResearchMicrosoft's research division, with laboratories on several continents, whose scientists have produced influential work on speech, vision and language AI.
MMMUA test for AI models that understand pictures as well as text: 11,500 college-level questions across 30 subjects, each built around one or more images such as a diagram, a musical score or a chemical structure.
MNISTA famous collection of 70,000 small images of handwritten digits used to test image-recognition programs. Getting a model to read MNIST digits is the 'hello world' of machine learning.
MobileNetA family of compact image-recognition networks from Google, designed to run quickly on phones and other small devices with little loss of accuracy.
model cascadeA chain of AI models ordered from cheapest to most capable: each request goes to the cheap one first and moves up the chain only if the answer does not look good enough.
multimodal AIAI that can take in or produce more than one kind of information, such as text, images, sound and video, so you can show it a photo and ask questions about it.
multimodal large language modelAn AI language model that can also take in — and sometimes produce — pictures, sound or video, so you can show it a photo or speak to it, not just type.
multimodal neuronA unit inside an AI vision model that responds to one idea however it appears — a photo, a drawing or the written name — much like the 'Halle Berry neuron' found in a human brain.
music generationUsing AI to compose or produce music — from melodies written as notes to complete songs with vocals — often from a short text description like 'upbeat jazz with saxophone'.
neocognitronA 1980 neural network for recognizing handwritten characters, modeled on the brain's visual system. Its stacked layers of pattern detectors were the blueprint for the networks used in modern image recognition.
network dissectionA method for finding out what individual units inside an image-recognition network respond to, such as dogs, windows, grass or stripes, by comparing where each unit lights up with pictures labeled by people.
neural audio codecA neural network that squeezes sound into a very compact stream of numbers and rebuilds it, used both to send audio cheaply and to let AI models 'write' speech and music piece by piece.
occlusion sensitivityFinding out which parts of an image matter to a model by covering them up one patch at a time and watching how much the model's answer changes.
occupancy gridA robot's map of the world drawn as a grid of small squares or cubes, each marked with the chance that something solid is there, so the robot knows where it can safely go.
odometryEstimating how far and in which direction a robot or vehicle has moved by adding up its own motion, such as wheel turns or how camera images shift, like counting your steps.
OSWorldA test in which AI agents must use real desktop computers — clicking, typing and switching between apps such as spreadsheets, browsers and file managers — to finish everyday office tasks.
Seymour PapertA South African-born mathematician who co-wrote 'Perceptrons', a critique of early neural networks, with Marvin Minsky, and invented the children's programming language Logo.
pattern recognitionThe ability of a computer, or a brain, to spot regularities in data and sort things into categories, such as recognizing handwritten letters, faces or spoken words.
π0A general-purpose AI model for controlling robots, released in 2024 by the start-up Physical Intelligence, that sees through cameras, follows typed or spoken instructions and moves robot arms deftly enough to fold laundry.
poolingA step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions.
proprioceptionA robot's sense of its own body: where its joints are, how fast they are moving and how hard its motors are pushing, measured by internal sensors rather than cameras.
prototypical part networkAn image classifier that explains each decision by pointing to parts of the picture that resemble typical examples it learned — this part looks like that part of a known bird.
Alec RadfordAn American AI researcher who led OpenAI's first GPT language models and its CLIP and Whisper systems, key steps toward today's chatbots and multimodal AI.
receptive fieldThe patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to.
recurrent neural networkA neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood.
recurrent processing theoryA theory that we consciously see something only when signals in the brain's visual areas loop back on themselves, not when they merely sweep forward once.
Raj ReddyAn Indian-American computer scientist who pioneered computer speech recognition and large AI systems at Carnegie Mellon University, and shared the 1994 Turing Award.
remote biometric identificationUsing AI to recognize people at a distance — usually by matching faces in camera footage against a database — without their taking part. The EU strictly limits live police use of it in public places.
ResNetA family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI.
saliency mapA heat map laid over an image or text that highlights which parts most influenced an AI model's output — or, in vision science, which parts of a scene would most catch a viewer's eye.
score distillation samplingA trick for creating 3D objects from a text description by repeatedly asking a 2D image generator whether pictures of the object, seen from many angles, look right — and adjusting the object until they do.
Oliver SelfridgeA British-born American computer scientist who pioneered teaching machines to recognize patterns; his 1959 'Pandemonium' model did it through layers of simple, competing detectors.
sensor fusionCombining readings from several different sensors, such as cameras, radar, lidar, GPS and motion sensors, to get a more accurate and reliable picture than any one sensor gives alone.
Siamese networkA matching machine built from twin copies of one neural network: each copy turns one of two inputs, say two signatures or two face photos, into a summary, and the two summaries are compared.
SiLUA smooth activation rule that multiplies each value by a soft on/off switch computed from the value itself. It is close to ReLU but curved, and is used in many modern models.
simultaneous localization and mappingHow a robot, drone or headset builds a map of an unfamiliar place while at the same time working out where it is on that map, using its cameras or laser scanners.
SmoothGradA trick for making explanation heat maps less speckled: compute the map for many noisy copies of the input and average them, so random flicker cancels out.
squeeze-and-excitation networkAn image-network design whose add-on blocks let the network decide, for each picture, which of its internal feature channels to turn up and which to turn down.
streamingSending an AI's answer piece by piece while it is still being written, so the words appear on screen one after another instead of all at once at the end.
strideHow far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
style transferRepainting a photo in the style of an artwork — your street as Van Gogh might have painted it — by keeping the photo's content and borrowing the painting's colors, strokes and textures.
super-resolutionMaking a small or blurry image larger and sharper by having AI fill in plausible fine detail, rather than just stretching the pixels.
tactile sensingGiving robots a sense of touch, with sensors in fingertips or skin that feel pressure, texture, vibration and slipping, so they can handle objects gently and securely.
testing with concept activation vectorsA method for asking whether a human idea, such as 'wheels', matters to an image model's decision, such as 'car', by finding the direction for that idea inside the model and measuring its effect.
text-to-image modelAn AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds.
text-to-video modelAn AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'.
time delay neural networkAn early network for speech that applied the same small pattern detector at every moment of a sound, so it could recognize a sound wherever it occurred in time.
tokenThe small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens.
transformerThe design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
transposed convolutionAn operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image.
U-NetAn image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost.
universal adversarial perturbationA single faint pattern that, added to almost any image, makes a neural network misclassify it — one key that opens many locks.
VeoGoogle's video-generating AI, which since 2025 creates short clips complete with matching sound and dialogue from a written prompt.
VGGNetAn influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.
virtual assistantSoftware you speak or type to, like Siri, Alexa or Google Assistant, that answers questions and does simple jobs such as setting timers, playing music or controlling smart-home devices.
vision-language modelAn AI model that understands images and text together — it can describe a photo, answer questions about a chart or find the picture that matches a sentence.
vision-language-action modelAn AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
Viterbi algorithmA method that finds the likeliest hidden sequence behind some observations — such as each word's grammatical role in a sentence — by keeping only the best path to each point instead of trying every combination.
voice agentAn AI system you talk to out loud, for example on a phone call, that listens, answers in a natural-sounding voice and can carry out tasks such as booking appointments or checking an order.
voice cloningUsing AI to copy a specific person's voice from recordings, so a computer can say new things that sound like them — useful for dubbing and accessibility, and abused in scams.
whole brain emulationCopying a particular person's brain into a computer in such detail, down to individual neurons and their connections, that the simulation would think and behave like that person.
zero-shot learningDoing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training.