clean-label poisoning
In plain wordsA sneaky form of data poisoning in which every planted training example is labeled correctly, so reviewers see nothing wrong, yet the examples are crafted to make the model fail.
A data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations. attack in which the poisoned training examples carry correct labels, so label auditing cannot catch them. Each poison is perturbed, often imperceptibly, so that its internal representation collides with a chosen target input (feature collision), or so that training on it pushes the model's weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. toward misclassifying the target (gradient matching). A clean-label backdoorbackdoor attack, n.: Secretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time. instead makes trigger-bearing examples hard to classify from their content, forcing the model to rely on the trigger. Clean-label attacks matter wherever data is scraped and labeled automatically or by trusted pipelines, because a plausibly labeled image or document passes inspection.
Fifty clean-label poisons, each an ordinary-looking dog photo, made the model call the target fish image a dog.
Origin
Introduced by Ali Shafahi and colleagues at the University of Maryland in 'Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks' (NeurIPS 2018). Alexander Turner, Dimitris Tsipras and Aleksander Madry extended the idea to backdoors, and gradient-matching attacks such as Witches' Brew (Jonas Geiping and colleagues, 2020) scaled it to ImageNet.
Related
- data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
- backdoor attackbackdoor attack, n.: Secretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time.
- NightshadeNightshade, prop. n.: A free tool for artists that subtly alters their images so that AI image generators trained on them without permission learn the wrong things — associating 'dog' with pictures of cats, say.
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
Sources
- Shafahi, A., Huang, W. R., Najibi, M., Suciu, O., Studer, C., Dumitras, T. & Goldstein, T. (2018). Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks. NeurIPS 2018.
Mentioned in
backdoor attackbackdoor attack, n.: Secretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time., data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations., NightshadeNightshade, prop. n.: A free tool for artists that subtly alters their images so that AI image generators trained on them without permission learn the wrong things — associating 'dog' with pictures of cats, say.