data leakage
In plain wordsA mistake in building a model that lets it peek at clues it will not have in real use, such as hints about the very answer it must predict, so it shines in testing and then disappoints.
A defect in how a model is built or evaluated: some signal reaches training, feature construction or model selection that no deployed system could have at decision time, so test results flatter the model. In target leakage an input effectively encodes the answer, often because it is recorded only after the outcome, such as a 'treatment given' field in a model meant to predict diagnosis. The other main route is a test settest set, n.: A set of examples kept completely apart from training and used at the end to measure how well a model works on data it has never seen, like a sealed final exam. that is not truly separate: the same patient, user or session on both sides of a split, duplicate or near-duplicate records, random splits of a time series that let the model see the future (look-ahead bias), or preprocessing, feature selectionfeature selection, n.: Choosing which of the available input measurements a model should use and dropping the rest, so the model becomes simpler, faster and often more accurate. and hyperparameter tuninghyperparameter tuning, n.: Trying out different settings for a model's training, such as learning speed or model size, and keeping the combination that performs best on data held back for checking. carried out before the data were split. Prevention relies on splitting first and fitting every preprocessing step inside the training data, on grouped and time-ordered cross-validationcross-validation, n.: A fair way to test a model when data are limited: split the data into several parts, train on all but one, test on the part left out, and rotate so every part takes a turn., on deduplication, and on asking of every feature whether it would be known at prediction time. In large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. evaluation, the analogous problem of test items appearing in training data is called data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability..
The 0.99 AUC was data leakage: the cancellation-reason field is only filled in after a customer has churned.
Origin
Practitioners had long met the problem in data-mining competitions such as the KDD Cup. Shachar Kaufman, Saharon Rosset and Claudia Perlich formalized it in 'Leakage in Data Mining: Formulation, Detection, and Avoidance' (KDD 2011). In 2023 Sayash Kapoor and Arvind Narayanan documented leakage-driven errors across hundreds of published studies that used machine learning in many scientific fields.
Usage note
Not to be confused with data leakage in information security (unauthorized disclosure of sensitive data) or with training data extractiontraining data extraction, n.: Getting an AI model to repeat, word for word, pieces of the data it was trained on, which may include someone's private contact details or copyrighted text. attacks on models.
Related
- test settest set, n.: A set of examples kept completely apart from training and used at the end to measure how well a model works on data it has never seen, like a sealed final exam.
- cross-validationcross-validation, n.: A fair way to test a model when data are limited: split the data into several parts, train on all but one, test on the part left out, and rotate so every part takes a turn.
- data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability.
- overfittingoverfitting, n.: When a model learns its training examples too well — quirks and noise included — so it does great on data it has seen but poorly on anything new, like a student who memorizes answers without understanding.
- feature engineeringfeature engineering, n.: Turning raw data into the inputs a model learns from, using knowledge of the problem; for example, converting a timestamp into 'hour of day' and 'is it a weekend'.
- training–serving skewtraining–serving skew, n.: A mismatch between how data is prepared when a model is trained and when it is used for real, which makes the live model quietly worse than its tests promised.
- generalizationgeneralization, n.: How well a model copes with new examples it has never seen, rather than just memorizing its training data, like a student who understands the method, not just the answers.
Sources
- Kaufman, S., Rosset, S. & Perlich, C. (2011). Leakage in Data Mining: Formulation, Detection, and Avoidance. Proc. KDD 2011.
- Kapoor, S. & Narayanan, A. (2023). Leakage and the Reproducibility Crisis in Machine-Learning-Based Science. Patterns 4(9).
Mentioned in
overfittingoverfitting, n.: When a model learns its training examples too well — quirks and noise included — so it does great on data it has seen but poorly on anything new, like a student who memorizes answers without understanding., test settest set, n.: A set of examples kept completely apart from training and used at the end to measure how well a model works on data it has never seen, like a sealed final exam., training settraining set, n.: The examples a machine-learning model learns from, like the worked problems a student practices on, kept separate from the examples later used to check how well it learned., AutoMLAutoML, n.: Software that automates the trial-and-error parts of building a machine-learning model, such as choosing the algorithm, its settings and the data-preparation steps, so less expert effort is needed., CatBoostCatBoost, prop. n.: A free gradient-boosting library from the tech company Yandex, known for handling categories such as city names or product types directly, without first converting them into numbers by hand., cross-validationcross-validation, n.: A fair way to test a model when data are limited: split the data into several parts, train on all but one, test on the part left out, and rotate so every part takes a turn., data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability., data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills., feature engineeringfeature engineering, n.: Turning raw data into the inputs a model learns from, using knowledge of the problem; for example, converting a timestamp into 'hour of day' and 'is it a weekend'., feature scalingfeature scaling, n.: Putting input measurements on comparable scales, so that income in dollars and age in years, say, both span similar ranges and no feature dominates just because of its units., feature selectionfeature selection, n.: Choosing which of the available input measurements a model should use and dropping the rest, so the model becomes simpler, faster and often more accurate., imputationimputation, n.: Filling in missing values in a dataset with sensible estimates, such as the typical value or a prediction from the other columns, so the data can be analyzed or used for training., KaggleKaggle, prop. n.: A popular website, owned by Google, where people compete to build the most accurate prediction models on shared datasets, publish their code and data, and learn data science., machine learning pipelinemachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way., one-hot encodingone-hot encoding, n.: Turning a category such as 'red', 'green' or 'blue' into a row of zeros with a single one marking which category it is, so that a model can use it as numbers., reproducibilityreproducibility, n.: Whether other people, or the same people later, can get the same results by rerunning an AI experiment. Many published AI results have proved hard to reproduce, which makes real progress harder to judge., SMOTESMOTE, abbr.: A technique for lopsided data that invents extra examples of the rare category by blending each rare example with one of its nearest rare neighbors, so a model sees more of it during training., time series forecastingtime series forecasting, n.: Predicting future values of something measured over time, such as sales, electricity demand, prices or rainfall, from its past values and related information., Arvind NarayananArvind Narayanan, prop. n.: An Indian-American computer scientist at Princeton who studies AI's real-world effects and argues that AI is a 'normal technology' whose impact will unfold gradually., feature storefeature store, n.: A shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live., training–serving skewtraining–serving skew, n.: A mismatch between how data is prepared when a model is trained and when it is used for real, which makes the live model quietly worse than its tests promised.