universal adversarial perturbation
In plain wordsA single faint pattern that, added to almost any image, makes a neural network misclassify it — one key that opens many locks.
One fixed perturbation δ, small under a chosen norm and computed once for a model rather than for each input, that makes the model misclassify most of the inputs it is added to. It is computed by iterating over training examples, accumulating the extra perturbation needed to fool each one, and projecting the running total back onto the norm ball. Universal perturbations often transfer across architectures, suggesting that models share correlated vulnerable directions in input space. Because one precomputed pattern suffices, they enable attacks without per-input computation; the idea foreshadowed physical patchesadversarial patch, n.: A printed sticker or picture designed so that, when it appears in a camera's view, an AI vision system misreads the scene — seeing a toaster, say, or missing a person. and, for language models, universal adversarial suffixesuniversal adversarial suffix, n.: A string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them. that work across many prompts.
A universal perturbation of ℓ∞ norm 10/255 fooled the classifier on 80% of validation images.
Origin
Introduced by Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi and Pascal Frossard in 'Universal Adversarial Perturbations' (arXiv October 2016; CVPR 2017).
Related
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
- adversarial patchadversarial patch, n.: A printed sticker or picture designed so that, when it appears in a camera's view, an AI vision system misreads the scene — seeing a toaster, say, or missing a person.
- universal adversarial suffixuniversal adversarial suffix, n.: A string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them.
- evasion attackevasion attack, n.: An attack on an already-trained AI system in which the attacker shapes an input — an email, a file, an image — so the system misjudges it, letting spam or malware slip through.
- black-box attackblack-box attack, n.: An attack on an AI model by someone who cannot see inside it and can only send it inputs and watch what comes out — like picking a lock by feel.
Sources
- Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O. & Frossard, P. (2017). Universal Adversarial Perturbations. CVPR 2017. arXiv:1610.08401.
Mentioned in
adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon., adversarial patchadversarial patch, n.: A printed sticker or picture designed so that, when it appears in a camera's view, an AI vision system misreads the scene — seeing a toaster, say, or missing a person., universal adversarial suffixuniversal adversarial suffix, n.: A string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them.