data-centric AI
In plain wordsAn approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model.
An approach to machine learning that puts most engineering effort into the dataset rather than the model architecture. In the recipe Andrew Ng popularized, a reasonable model stays fixed while the team improves the data methodically, through consistent labeling guidelines, correction of label noiselabel noise, n.: Mistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems., targeted collection of missing or rare cases, data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly., deduplicationdeduplication, n.: Removing repeated copies of the same or nearly the same content from a dataset, so a model does not over-learn whatever happens to be copied many times online. and documentation. Proponents argue that in many applied settings, especially those with small datasets such as manufacturing inspection or medical imaging, label consistency and coverage limit performance more than model choice, and that data work should be made systematic rather than ad hoc. The framing contrasts with model-centric research that benchmarks new architectures on fixed datasets. In large-model development its themes reappear in quality filteringquality filtering, n.: Automatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk., curated instruction datainstruction dataset, n.: A collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant. and research on data mixtures, although the label itself is used mostly in applied machine learning.
Instead of trying a bigger network, we went data-centric and relabeled the 600 most ambiguous images.
Origin
Popularized by Andrew Ng in 2021 through a March 2021 talk contrasting model-centric and data-centric AI, a Data-Centric AI Competition and a NeurIPS 2021 workshop. Its concerns echo long-standing data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills. research and Nithya Sambasivan and colleagues' 2021 study of 'data cascades' in high-stakes AI.
Usage note
Some researchers object that the label repackages old practices — data cleaning, active learning, careful annotation — as a new paradigm; others value it for giving data work academic credit and venues such as datasets-and-benchmarks conference tracks.
Related
- data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills.
- data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
- label noiselabel noise, n.: Mistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems.
- data labelingdata labeling, n.: The work of attaching correct answers to examples — often done by large paid workforces, sometimes by experts — so that AI systems can learn from them or be tested against them.
- data flywheeldata flywheel, n.: A cycle in which a product's users generate data that improves its AI, the better AI attracts more users, and those users generate still more data.
- active learningactive learning, n.: A way to train a model with fewer labeled examples by letting it choose which unlabeled examples a person should label next, usually the ones it is most unsure about.
Sources
- Sambasivan, N. et al. (2021). 'Everyone wants to do the model work, not the data work': Data Cascades in High-Stakes AI. CHI 2021.
Mentioned in
Andrew NgAndrew Ng, prop. n.: A British-born American computer scientist whose online courses have taught machine learning to millions; he also co-founded Google's deep learning team, Google Brain, and the online-course company Coursera., data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills., data flywheeldata flywheel, n.: A cycle in which a product's users generate data that improves its AI, the better AI attracts more users, and those users generate still more data.