Data & Datasets

Datasets, labels, corpora and provenance, and the problems data brings. 62 entries live here, and 143 from other domains touch it.

Essential

dataRecorded facts, measurements, text, images or sounds that a computer can store and process. AI systems learn their skills from large amounts of data rather than from rules written by hand.
datasetAn organized collection of examples gathered for a purpose — such as photos with their captions, or questions with answers — used to teach or test an AI system.
labelThe correct answer attached to an example in a dataset — 'cat' on a photo of a cat, 'positive' on an upbeat review — that a learning system is trained to predict.
synthetic dataData created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
training dataThe examples an AI system learns from. Whatever patterns, skills, gaps and biases are in the training data tend to show up in the finished system.

All other entries

Amazon Mechanical TurkA website Amazon opened in 2005 that acts as a hiring hall for tiny online jobs, such as labeling images for AI, which registered workers pick up and finish for a few cents each.
annotationAdding information to raw data by hand or with software — marking objects in photos, tagging names in sentences, transcribing speech or ranking chatbot answers — so machines can learn from it.
BookCorpusA collection of about 11,000 free self-published novels gathered from the web in 2015, used to train the first GPT model and BERT despite problems with consent and duplicates found later.
Books3A collection of nearly 200,000 pirated e-books, assembled in 2020 and used to train several AI language models, which became central to lawsuits by authors over AI training.
Brown CorpusThe first large collection of English text stored on computer for research: about a million words from 500 American publications of 1961, assembled at Brown University in the 1960s.
C4A large collection of English web text, cleaned up from a 2019 web archive snapshot by Google researchers, that became a standard ingredient for training language models.
Cohen's kappaA score for how much two people labeling the same items agree beyond what lucky guessing would produce: 1 means perfect agreement, 0 means no better than chance.
Common CrawlA nonprofit's free archive of the public web, begun in 2008 and now refreshed with snapshots of billions of pages every month or two. Most large language models learned much of what they know from text taken from it.
corpusA large collection of written or spoken language, gathered so that people or computers can study it or learn from it — like a library assembled for a language model.
covariate shiftA kind of mismatch where the inputs an AI system sees change — say, photos from a new camera — while the right answer for any given input stays the same.
crowdsourcingGetting many people over the internet to each do a small piece of a big job, such as labeling pictures for a few cents apiece, with anyone free to join in.
data attributionWorking out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it.
data cleaningFinding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
data contaminationWhen the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability.
data flywheelA cycle in which a product's users generate data that improves its AI, the better AI attracts more users, and those users generate still more data.
data labelingThe work of attaching correct answers to examples — often done by large paid workforces, sometimes by experts — so that AI systems can learn from them or be tested against them.
data licensingThe legal permissions that say who may use a dataset or a body of content, for what purposes and on what terms — including whether it may be used to train AI.
data pipelineThe chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material.
data provenanceThe documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system.
data qualityHow well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills.
data wallThe worry that AI companies will run out of new human-written text to train ever-larger models on, because the useful public internet has already been largely used up.
data-centric AIAn approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model.
datasheet for datasetsA standard document that travels with a dataset, describing why it was made, what is in it, how it was collected and labeled, and what it should and should not be used for.
deduplicationRemoving repeated copies of the same or nearly the same content from a dataset, so a model does not over-learn whatever happens to be copied many times online.
distribution shiftWhen the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested.
FineWebA huge, free collection of cleaned-up English web text — around 15 trillion word fragments — released by Hugging Face in 2024 with a detailed public account of how it was built.
golden datasetA small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes.
ground truthThe reference answers treated as correct when checking or training an AI system — for example, the confirmed diagnoses that a medical-imaging model's guesses are compared against.
instruction datasetA collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant.
inter-annotator agreementA measure of how often different people labeling the same data give the same answer. Low agreement warns that the task is unclear or the labels are unreliable.
label noiseMistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems.
label shiftA kind of mismatch where the mix of answers changes — say, a disease becomes far more common during an outbreak — while each individual case still looks the same as before.
LAION-5BA free list of nearly six billion links to online images paired with their captions, released in 2022 and used to train image generators such as Stable Diffusion.
language identificationAutomatically working out which language a piece of text or speech is in — a basic sorting step when building multilingual AI training data from the web.
locality-sensitive hashingA trick for finding look-alikes in a huge collection without comparing every pair: items get short codes designed so that similar items tend to share a code, and only items sharing one are compared.
long tailThe huge number of rare things — uncommon words, unusual road situations, obscure facts — that each appear seldom but together make up a large share of what an AI system meets.
low-resource languageA language with little digital text, recorded speech or language tools available — often one spoken by millions — which makes AI systems much weaker in it than in English.
MinHashA shortcut for spotting near-copies among millions of documents: each document is boiled down to a short fingerprint, and similar documents end up with similar fingerprints.
model cardA short document published with an AI model explaining what it is for, how it was trained and tested, how well it works for different groups of people, and its known limits.
model collapseThe gradual decline of AI models trained on the output of earlier AI models: rare details vanish first, then quality and variety erode, like a photocopy of a photocopy.
MS MARCOA large Microsoft dataset of real Bing search questions paired with web passages that answer them, used to train and test most modern AI search models.
opt-outA way for creators, websites or individuals to say 'do not use my work or data to train AI' — instead of being asked for permission first, which would be opt-in.
out-of-distributionDescribes inputs that differ from anything an AI system saw in training — like a model trained on daytime photos facing a night scene — where its answers become less reliable.
Penn TreebankA collection of English text, best known for its million words of newspaper articles, in which experts marked each word's grammatical role and drew each sentence's structure as a tree; it trained and tested a generation of language software.
personally identifiable informationInformation that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person.
pretraining dataThe enormous body of text, code and other material — mostly from the web — that a large AI model first learns from, before it is tuned to follow instructions or chat.
quality filteringAutomatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk.
robots.txtA small public file in which a website posts its rules for automated visitors, such as search-engine and AI crawlers, listing areas they are asked to stay out of. Obeying it is voluntary.
sampling biasA distortion that arises when the data collected over-represents some kinds of people or situations and under-represents others, so conclusions drawn from it don't hold for everyone.
Self-InstructA technique in which a language model writes its own practice tasks and answers, starting from a handful of human examples; the results are filtered and used to train it to follow instructions.
shortcut learningWhen an AI system gets right answers for the wrong reasons — noticing which hospital an X-ray came from rather than signs of disease — so it fails once that accidental clue disappears.
The PileA large free collection of English text released in 2020 by the volunteer group EleutherAI, mixing 22 sources from books and science papers to code, used to train early open language models.
The StackA large free collection of computer source code from public repositories, selected mainly for permissive licenses and released from 2022 onward by the BigCode project for training coding models.
web crawlerA program that automatically travels the web, following links from page to page and downloading what it finds — used by search engines, archives and AI companies to gather data.
web scrapingUsing software to automatically copy information from websites — text, prices, images — in bulk, which is how much of the data used to train AI was gathered.
WebTextThe private collection of about eight million web pages, chosen because Reddit users had linked to them, that OpenAI used in 2019 to train GPT-2.
WordNetA hand-made map of English vocabulary in which words sharing a meaning are bundled together and the bundles are linked — recording, say, that a dog is a kind of animal and a wheel is part of a car.

Also relevant

143

active learningA way to train a model with fewer labeled examples by letting it choose which unlabeled examples a person should label next, usually the ones it is most unsure about.
AI colonialismThe criticism that the AI industry repeats patterns of colonial empires, taking data, labor, land and resources from poorer places and peoples while profits and control stay with a few rich companies and countries.
AI copyright litigationThe many lawsuits in which authors, artists, news organizations, record labels and others accuse AI companies of copying their work without permission to build or run AI models.
AI Incident DatabaseA free public online collection of reports about real cases in which AI systems caused or nearly caused harm, kept so that builders and regulators can learn from past failures.
AI supply chain securityProtecting everything an AI system is built from — downloaded models, datasets, code libraries and tools — against tampering, so a poisoned or fake part cannot sneak into the finished product.
algorithmic biasUnfairness in software that judges or serves people: it consistently produces worse results for some groups, such as women, older people or ethnic minorities, often because it learned from skewed or incomplete records.
annealingThe last stretch of training, when the model's learning is slowed to a stop — often while it is fed a small helping of the best data — so it settles into its final, sharpest version.
anomaly detectionAutomatically spotting the rare data points that don't fit the usual pattern, such as a fraudulent card payment, a machine about to fail or an intruder on a network.
association rule learningFinding 'if this, then often that' patterns in large records of transactions, such as noticing that shoppers who buy pasta and tomatoes often buy parmesan too.
bag of wordsA simple way of turning text into numbers by counting how often each word appears and ignoring word order — like describing a recipe only by its shopping list.
Bartz v. AnthropicA US lawsuit by book authors against the AI company Anthropic that ended in a record $1.5 billion settlement, after a judge ruled that training on purchased books was legal but keeping pirated copies was not.
benchmarkA fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
big dataData that piles up too fast, in too many forms or in too great a quantity for one ordinary computer to store and study, and the methods invented to cope with it.
biometric dataInformation about your body or behavior — your face, fingerprints, voice, eyes or the way you walk — measured by technology and used to recognize who you are or to guess things about you.
bounding boxThe rectangle a vision system draws around an object it has found, stored as a handful of numbers giving the rectangle's position and size.
byte-pair encodingA popular way to choose the word pieces an AI breaks text into: start from single characters and repeatedly glue together the pair that occurs most often, until the list reaches the desired size.
canary stringA unique marker planted in data to trace what an AI model has absorbed: a code in test files so they can be kept out of training, or a fake secret hidden in training data to measure memorization.
chunkingCutting long documents into smaller pieces before storing them for AI search, so the system can find and hand the model just the relevant part instead of a whole manual.
class imbalanceWhen one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters.
clean-label poisoningA sneaky form of data poisoning in which every planted training example is labeled correctly, so reviewers see nothing wrong, yet the examples are crafted to make the model fail.
click-through rate predictionEstimating how likely a particular person is to click on a particular ad, search result or recommended item: a prediction made billions of times a day to decide what each user is shown.
closed-world assumptionThe rule of treating whatever a system cannot confirm as false: if a library's catalog does not list a book, the library is assumed not to have it.
clusteringAutomatically sorting things into groups of similar items when nobody has said in advance what the groups should be, like sorting a drawer of odd socks into piles by color and size.
COCOA widely used collection of everyday photos in which people outlined and named the objects, described each picture in sentences and marked body joints; it is used to train and compare vision systems.
concept driftWhen the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold.
ConceptNetA free, crowd-built network of everyday facts — 'a knife is used for cutting', 'rain makes things wet' — in many languages, which AI systems use to make sense of ordinary situations.
contamination-resistant benchmarkA test built so that its questions are unlikely to have been seen by an AI during training — kept secret, refreshed often, or written after the model was made.
continued pretrainingTaking an already-trained AI model and resuming its broad, read-everything training on new material — a specialty such as medicine or law, a new language, or fresher data — rather than starting from scratch.
Kate CrawfordAn Australian scholar who studies the social, political and environmental costs of AI, co-founded the AI Now Institute, and wrote 'Atlas of AI'.
curriculum learningTeaching a model the way schools teach people: starting with easy examples and gradually moving to harder ones, instead of presenting everything in random order.
data augmentationCreating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly.
data driftWhen the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable.
data leakageA mistake in building a model that lets it peek at clues it will not have in real use, such as hints about the very answer it must predict, so it shines in testing and then disappoints.
data miningSearching large collections of data with computers to discover useful patterns, such as products often bought together or signs that a transaction may be fraudulent.
data mixtureThe recipe of different kinds of data a model is trained on — how much web text, code, books, math, science and other languages — which strongly shapes what the finished model is good at.
data poisoningSneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
data protectionLaws and practices that control how organizations collect, use, store and share information about people, and that give people rights over it, such as seeing it or having it deleted.
data scienceThe practice of collecting, cleaning, analyzing and interpreting data, using statistics, programming and machine learning, to answer questions and guide decisions.
data versioningKeeping a record of every version of the data used to train and test AI models, so that any past result can be traced back to exactly the data that produced it.
data-constrained scalingHow to keep improving AI models when the supply of fresh, high-quality training text runs out — for example by reusing the same data several times, which works well at first and then stops helping.
DatalogA small logic language for asking questions of a database with rules, including rules that refer to themselves, such as 'X is an ancestor of Y if X is a parent of an ancestor of Y'.
DBSCANA clustering method that finds groups as crowded regions of data separated by sparse ones, so it can discover oddly shaped groups and set stray points aside as noise.
description logicA family of precise logical languages for defining categories and how they relate, such as 'a parent is a person with at least one child', designed so that a computer's reasoning about them always finishes.
differential privacyA mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
digital twinA detailed virtual copy of a real machine, factory, robot or city, kept in sync with live sensor data and used to monitor it, test changes safely and predict problems.
dimensionality reductionSqueezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from.
domain adaptationAdjusting a model trained on one kind of data, such as daytime photos or one hospital's records, so that it still works on a different but related kind, such as night photos or another hospital's.
domain randomizationTraining a robot's AI in a simulation whose colors, lighting, weights, friction and other details are shuffled at random on every run, so that the real world holds few surprises it has not already practiced for.
EleutherAIAn open-science research group, begun by online volunteers in 2020, that released some of the first large language models anyone could download, along with the text collections used to train them.
exampleA single item of data used to train or test an AI model, such as one photo with its label, one email marked spam or not, or one question with its answer.
fair use and AI trainingThe unsettled US legal question of whether AI companies may copy books, articles, pictures and music to train their models without permission, under the rule that allows some unlicensed uses of copyrighted work.
feature engineeringTurning raw data into the inputs a model learns from, using knowledge of the problem; for example, converting a timestamp into 'hour of day' and 'is it a weekend'.
feature scalingPutting input measurements on comparable scales, so that income in dollars and age in years, say, both span similar ranges and no feature dominates just because of its units.
feature selectionChoosing which of the available input measurements a model should use and dropping the rest, so the model becomes simpler, faster and often more accurate.
feature storeA shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live.
federated learningTraining one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates.
feedback loopA cycle in which what a system produces comes back to shape its own future input, like a microphone howling near its loudspeaker. In AI, a model's predictions can shape the very data it later learns from.
Timnit GebruAn Ethiopian-born American computer scientist who exposed racial and gender bias in AI systems, co-wrote the 'stochastic parrots' paper, and founded an independent AI research institute.
General Data Protection RegulationThe European Union's main privacy law, in force since 2018, which controls how personal information is collected and used — including to train and run AI systems.
generative engine optimizationShaping web content so that AI chatbots and AI search features are more likely to find it, quote it and recommend it — the AI-era version of search engine optimization.
ghost workThe hidden human labor behind AI: people, often poorly paid and far away, who label data, check outputs and handle cases the software cannot, while the product appears fully automatic.
hierarchical clusteringA way of grouping data that builds a family tree of clusters, joining the most similar items step by step, so you can cut the tree at whatever level of detail you need.
human evaluationJudging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
i.i.d.Short for 'independent and identically distributed': every data point is produced the same way, and none influences any other — like repeated rolls of one fair die.
ImageNetA huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom.
implicit feedbackClues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give.
imputationFilling in missing values in a dataset with sensible estimates, such as the typical value or a prediction from the other columns, so the data can be analyzed or used for training.
influence functionA mathematical shortcut for estimating how much a single training example shaped a model's behavior, for instance how a prediction would change if that example had been left out, without retraining the model.
information extractionAutomatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database.
information retrievalThe study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up.
instruction tuningTraining a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
inverted indexA lookup table that lists, for every word, all the documents containing it — like the index at the back of a book — so a search engine can find matches almost instantly.
Jaccard similarityA simple score for how much two collections overlap: the number of items they share divided by the number of distinct items in either. Identical sets score 1; sets with nothing in common score 0.
Jupyter notebookAn interactive document, opened in a web browser, that mixes runnable code with its results, charts and notes, so people can experiment step by step and share their work.
k-anonymityA rule for anonymizing tables of personal data: blur details such as age and postcode until every person matches at least k others, so no one can be singled out.
KDDThe leading yearly conference on data mining — finding useful patterns in large datasets — where many widely used machine learning tools were first presented.
knowledge baseA store of facts and rules that a program can reason with: like a database, but organized so that new conclusions can be worked out from it, not just looked up.
knowledge cutoffThe date after which an AI model knows nothing from its training, because its training data stops there. Without a web search tool, it cannot know about later events.
knowledge graphA huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over.
LAIONA mostly volunteer group, registered as a German nonprofit, that compiled free lists of billions of online pictures and their captions. Stable Diffusion and other image generators were trained on them.
Fei-Fei LiA Chinese-American computer scientist whose ImageNet project, millions of hand-labeled photos, gave deep learning the proving ground that launched its boom; she now champions 'human-centered' AI.
LibriSpeechA free collection of about 1,000 hours of volunteers reading English audiobooks, matched with the book text; it became the standard yardstick for speech-recognition systems.
LLM groomingFlooding the internet with propaganda written mainly for machines rather than people, hoping AI chatbots will soak it up and repeat it to their users.
llms.txtA short guide file that a website puts at a standard address to help AI assistants find and understand its most useful pages.
local differential privacyA privacy method in which each person's device scrambles their data with random noise before sending it anywhere, so the company collecting it never sees anyone's true answer but can still estimate overall trends.
machine learning pipelineAn automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way.
machine unlearningErasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
MathlibA huge shared library of mathematics written in the Lean language, in which every definition and proof has been checked by computer. Mathematicians keep extending it, and AI proof systems are trained on it.
membership inference attackAn attack that tries to tell whether a particular person's record, or a particular document, was part of an AI model's training data — which can itself reveal private facts.
memorizationWhen an AI model stores specific pieces of its training data — a paragraph, a phone number, an image — closely enough to reproduce them, instead of only learning general patterns.
Message Understanding ConferenceA series of US government-funded contests held from 1987 to 1998 in which research teams built programs to pull key facts, such as who did what, where and when, out of news and military messages.
mid-trainingA middle stage in building a language model, between its broad initial training and its tuning into an assistant: it keeps learning the same way, but from a smaller, carefully chosen diet of math, code, reasoning or long documents.
Margaret MitchellAn American computer scientist who proposed 'model cards' for documenting AI systems, co-led Google's Ethical AI team, and is chief ethics scientist at Hugging Face.
MLOpsThe working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself.
MNISTA famous collection of 70,000 small images of handwritten digits used to test image-recognition programs. Getting a model to read MNIST digits is the 'hello world' of machine learning.
model hubAn online library where trained AI models are published so that others can find, download and reuse them instead of training their own from scratch.
model inversionUsing a trained AI model's answers to work backwards to private information it learned from — for example, rebuilding a recognizable picture of someone's face from a face-recognition system.
MTEBA large public test suite that scores AI models which turn text into lists of numbers on many different jobs, such as searching, grouping, classifying and matching translations, so people can compare them on one leaderboard.
named-entity recognitionSoftware that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date.
National AI Research ResourceA US program to give university researchers, students and small companies shared access to the costly computers, data and AI models that are otherwise mostly available to big technology firms.
Netflix PrizeA 2006–2009 contest in which Netflix offered a million dollars to anyone who could predict customers' movie ratings 10 percent better than its own system. It popularized machine-learning competitions and model blending.
NightshadeA free tool for artists that subtly alters their images so that AI image generators trained on them without permission learn the wrong things — associating 'dog' with pictures of cats, say.
offline reinforcement learningLearning a strategy entirely from a fixed collection of recorded experience — logs, past robot runs, medical records — without trying anything new in the real world while learning.
one-hot encodingTurning a category such as 'red', 'green' or 'blue' into a row of zeros with a single one marking which category it is, so that a model can use it as numbers.
ontologyA carefully defined vocabulary for a subject such as medicine, spelling out each kind of thing, how kinds nest inside one another, and which connections between them are allowed.
Open X-EmbodimentA large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them.
pandasA free Python library for loading, cleaning, combining and summarizing tables of data, like a programmable spreadsheet; it is a standard first tool for preparing data for machine learning.
point cloudA 3D picture made of many dots floating in space, each marking a spot on a surface, as captured by a laser scanner or worked out from photos.
preference dataRecords of which of two or more AI answers people (or other AIs) liked better, collected so that a model can learn what good answers look like.
principal component analysisA technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores.
privacyYour ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared.
privacy budgetThe total amount of privacy risk a data holder agrees to spend; each analysis or release uses up part of it, and once it is gone, no more answers can be given safely.
privacy-preserving machine learningThe family of techniques for training and using AI models while keeping the underlying personal or confidential data hidden — from adding careful noise to computing on encrypted data.
Resource Description FrameworkA web standard for writing facts as simple three-part statements — subject, relation, object — using web addresses as names, so that data from different sources can be merged automatically.
samplingPicking some items at random from a larger set or from a range of possibilities — surveying 1,000 voters, say, or letting an AI choose its next word by a weighted lottery.
Scale AIA company that pays large numbers of people to label examples, write model answers and grade chatbot replies, producing the data AI developers train and test on. Meta bought nearly half of it in 2025.
Segment Anything ModelA family of AI models from Meta that trace the exact outline of whatever you point to in a photo or video, whether by clicking on it, drawing a box around it or, in newer versions, naming it.
semantic webAn effort to label data on the web in a standard, precise way so that programs, not only human readers, can tell what it means and connect facts drawn from many different sites.
SentencePieceFree software from Google that learns how to split text into tokens straight from raw sentences in any language, treating spaces as ordinary characters, so no language-specific word splitting is needed.
SMOTEA technique for lopsided data that invents extra examples of the rare category by blending each rare example with one of its nearest rare neighbors, so a model sees more of it during training.
stemmingCutting words down to a common root so that variants count as the same word — 'connected', 'connecting' and 'connection' all become 'connect' — to help search and text analysis.
subliminal learningThe finding that an AI model can pass hidden traits — a favorite animal, or bad behavior — to another model through training data that seems to have nothing to do with them.
surveillance capitalismA business model in which companies collect huge amounts of data about people's behavior and use it to predict and influence what they will do, mainly to sell advertising.
tabular dataData arranged as a table, like a spreadsheet: each row is a case — a customer, a loan, a patient — and each column a measured property. Much business and scientific data looks like this.
tabular foundation modelA neural network pretrained on millions of made-up tables so that, given a new spreadsheet of labeled examples, it can predict the missing answers at once, without being retrained for that table.
teleoperationControlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen.
text and data mining exceptionA rule in many countries' copyright laws that lets people use computers to copy and analyze large amounts of text, images or data without asking permission — sometimes only for research, sometimes unless owners opt out.
tokenizationThe first step in feeding text to a language model: chopping it into tokens — whole words, word pieces or characters — and turning each into a number the model can work with.
tokenizerThe component that splits text into tokens and converts them to numbers before an AI model sees it, then turns the model's output numbers back into text.
training data extractionGetting an AI model to repeat, word for word, pieces of the data it was trained on, which may include someone's private contact details or copyrighted text.
training setThe examples a machine-learning model learns from, like the worked problems a student practices on, kept separate from the examples later used to check how well it learned.
training–serving skewA mismatch between how data is prepared when a model is trained and when it is used for real, which makes the live model quietly worse than its tests promised.
TRECA yearly contest, run by a US government standards agency since 1992, in which search-engine researchers test their systems on the same documents and questions, with people judging which results are actually relevant.
t-SNEA technique for drawing data that have many measurements per item as a flat map in which similar items land close together, widely used to look for groups in AI models' internal data or in genetic data.
UMAPA fast technique for turning data with many measurements per item into a flat map that keeps similar items together; often preferred to older methods because it is quicker and copes with more data.
vector databaseA database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools.
vocabularyThe fixed list of all the tokens a language model knows, each with its own number. Everything the model reads or writes must be built from pieces on this list.
Alexandr WangAn American entrepreneur who founded Scale AI, a major supplier of human-labeled data for training AI, and became Meta's chief AI officer in 2025.
weak supervisionTraining AI with labels that are cheap but imperfect, such as rough rules of thumb, partial tags or noisy guesses, instead of carefully hand-checked answers.
Web Ontology LanguageA web standard language for writing ontologies — precise definitions of categories and their relationships — so that software can check them for contradictions and infer new facts.
WikidataA free, shared collection of facts about millions of things — people, places, books, chemicals — kept by volunteers in the Wikipedia family and laid out so programs can use it as easily as people.
WordPieceGoogle's method for splitting words into smaller known pieces, used by BERT. An unfamiliar word such as 'unhappiness' might be broken into parts like 'un', '##happi' and '##ness'.
Zipf's lawThe pattern that word use is lopsided: in a big text the top word is roughly twice as common as the runner-up and ten times as common as the tenth, so a few words dominate and most are rare.