data provenance
In plain wordsThe documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system.
A record of the origin and custody of data — its sources, creators, collection dates and methods, license terms, and the transformations, filters and merges applied along the way — sufficient to trace a dataset or an individual example back to where it came from. In AI, provenance underpins legal compliance (copyright, consent, personal datapersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person.), reproducibility, diagnosis of model behavior and detection of data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability.. It is hard to maintain because training corpora are assembled from datasets that were themselves built from other datasets, with licenses often lost along the way: the Data Provenance Initiative's 2023 audit of more than 1,800 text datasets found license miscategorization rates of 50% or more. Practices include machine-readable dataset cards, lineage tracking in data pipelinesdata pipeline, n.: The chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material., content hashes and signed metadata. Regulation increasingly demands it: the EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models. requires providers of general-purpose modelsgeneral-purpose AI model, n.: The EU's legal name for the large, adaptable AI models behind chatbots and similar tools: models that can turn their hand to many different jobs and be built into many other products. to publish a summary of the content used for training.
Without provenance records we could not tell whether the forum data was collected before or after the site's opt-out.
Origin
'Provenance' comes from art and archives (French provenir, 'to come from'). Database researchers formalized data provenance and lineage from the late 1990s; Peter Buneman, Sanjeev Khanna and Wang-Chiew Tan's 2001 paper on 'why' and 'where' provenance is a landmark. The Data Provenance Initiative (Shayne Longpre and colleagues, 2023) brought the term to AI datasets.
Usage note
Distinguish data provenance (where training and evaluation data came from) from content provenance, which records the origin of individual media files, including AI-generated ones (see Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect.).
Related
- datasheet for datasetsdatasheet for datasets, n.: A standard document that travels with a dataset, describing why it was made, what is in it, how it was collected and labeled, and what it should and should not be used for.
- data licensingdata licensing, n.: The legal permissions that say who may use a dataset or a body of content, for what purposes and on what terms — including whether it may be used to train AI.
- data pipelinedata pipeline, n.: The chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material.
- model cardmodel card, n.: A short document published with an AI model explaining what it is for, how it was trained and tested, how well it works for different groups of people, and its known limits.
- opt-outopt-out, n.: A way for creators, websites or individuals to say 'do not use my work or data to train AI' — instead of being asked for permission first, which would be opt-in.
- data attributiondata attribution, n.: Working out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it.
- Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect.
Sources
- Buneman, P., Khanna, S. & Tan, W.-C. (2001). Why and Where: A Characterization of Data Provenance. ICDT 2001.
- Longpre, S. et al. (2023). The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. arXiv:2310.16787. arxiv.org/abs/2310.16787
Mentioned in
datasetdataset, n.: An organized collection of examples gathered for a purpose — such as photos with their captions, or questions with answers — used to teach or test an AI system., training datatraining data, n.: The examples an AI system learns from. Whatever patterns, skills, gaps and biases are in the training data tend to show up in the finished system., AI supply chain securityAI supply chain security, n.: Protecting everything an AI system is built from — downloaded models, datasets, code libraries and tools — against tampering, so a poisoned or fake part cannot sneak into the finished product., Content CredentialsContent Credentials, prop. n.: A tamper-evident 'nutrition label' attached to a photo, video or document recording where it came from — which camera or AI tool made it and how it was edited — that anyone can inspect., data licensingdata licensing, n.: The legal permissions that say who may use a dataset or a body of content, for what purposes and on what terms — including whether it may be used to train AI., data pipelinedata pipeline, n.: The chain of automated steps that collects raw data, cleans and transforms it, and delivers it in a usable form — the factory line that turns messy inputs into training material., data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations., data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills., feedback loopfeedback loop, n.: A cycle in which what a system produces comes back to shape its own future input, like a microphone howling near its loudspeaker. In AI, a model's predictions can shape the very data it later learns from., model cardmodel card, n.: A short document published with an AI model explaining what it is for, how it was trained and tested, how well it works for different groups of people, and its known limits., model collapsemodel collapse, n.: The gradual decline of AI models trained on the output of earlier AI models: rare details vanish first, then quality and variety erode, like a photocopy of a photocopy., Books3Books3, prop. n.: A collection of nearly 200,000 pirated e-books, assembled in 2020 and used to train several AI language models, which became central to lawsuits by authors over AI training., C2PAC2PA, abbr.: An industry group, and the technical standard it publishes, for attaching tamper-evident records of origin and editing history to digital photos, video, audio and documents., data attributiondata attribution, n.: Working out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it., data versioningdata versioning, n.: Keeping a record of every version of the data used to train and test AI models, so that any past result can be traced back to exactly the data that produced it., datasheet for datasetsdatasheet for datasets, n.: A standard document that travels with a dataset, describing why it was made, what is in it, how it was collected and labeled, and what it should and should not be used for., LAION-5BLAION-5B, prop. n.: A free list of nearly six billion links to online images paired with their captions, released in 2022 and used to train image generators such as Stable Diffusion.