certified robustness
In plain wordsA mathematical guarantee — not just a test result — that no change to an input within a set limit can alter a model's answer.
A guarantee, established by proof or sound computation rather than by attacking, that a model's prediction cannot change under any perturbation in a specified set, typically a norm ball of radius ε around a given input. Certificates come from exact verification (mixed-integer programming or satisfiability-modulo-theories solvers, feasible only for small networks), sound relaxations that bound a network's outputs (interval bound propagation, linear and convex relaxations), architectures with bounded Lipschitz constants, or probabilistic methods such as randomized smoothingrandomized smoothing, n.: A way to make a classifier provably harder to fool: ask it about many noisy copies of an input and take a majority vote, which small deliberate changes cannot easily flip.. Certified accuracy — the fraction of test points that are both correct and certified at radius ε — is a lower bound on true adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up., complementing attack-based estimates, which are upper bounds. Certified radii remain small relative to perceptible changes, and certification is costly for large models.
The model's certified accuracy at ℓ2 radius 0.5 is 61%, against 74% empirical robust accuracy.
Origin
Formal verification of neural networks predates deep learning's boom; Guy Katz and colleagues' Reluplex solver (2017) was a landmark. 'Certified defenses' were introduced in 2018 by Aditi Raghunathan, Jacob Steinhardt and Percy Liang (semidefinite certificates) and, concurrently, Eric Wong and J. Zico Kolter (convex outer bounds); randomized smoothing (2019) scaled certificates to ImageNet.
Usage note
A certificate covers only its stated perturbation set and input; it says nothing about patches, semantic edits or other perturbation types unless those are certified separately.
Related
- adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up.
- randomized smoothingrandomized smoothing, n.: A way to make a classifier provably harder to fool: ask it about many noisy copies of an input and take a majority vote, which small deliberate changes cannot easily flip.
- adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
- threat modelthreat model, n.: A clear statement of who might attack a system, what they want, what they know and what they can do — the yardstick for judging whether a defense is good enough.
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
- adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
Sources
- Raghunathan, A., Steinhardt, J. & Liang, P. (2018). Certified Defenses against Adversarial Examples. ICLR 2018.
- Wong, E. & Kolter, J. Z. (2018). Provable Defenses against Adversarial Examples via the Convex Outer Adversarial Polytope. ICML 2018.
Mentioned in
adversarial machine learningadversarial machine learning, n.: The study of how machine-learning systems can be fooled, corrupted or spied on by someone working deliberately against them, and of how to build systems that hold up., adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up., adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them., robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it., Lipschitz continuityLipschitz continuity, n.: A guarantee that a function never changes too abruptly: its output can move at most a fixed multiple of how far its input moves., randomized smoothingrandomized smoothing, n.: A way to make a classifier provably harder to fool: ask it about many noisy copies of an input and take a majority vote, which small deliberate changes cannot easily flip.