Skip to content
Last Dictionary
Plane
Security & MisuseDeep LearningMathematics & Statistics

randomized smoothing

nounAlso: smoothed classifier

In plain wordsA way to make a classifier provably harder to fool: ask it about many noisy copies of an input and take a majority vote, which small deliberate changes cannot easily flip.

  1. A certified robustnesscertified robustness, n.: A mathematical guarantee — not just a test result — that no change to an input within a set limit can alter a model's answer. technique that wraps a base classifierclassifier, n.: A program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers. f in a smoothed classifier g: g adds isotropic Gaussian noise of standard deviation σ to the input many times and outputs the class f assigns most often to the noisy copies. If, under that noise, the top class has probability at least p_A and the runner-up at most p_B, then g's prediction provably cannot change under any ℓ2 perturbation smaller than the radius R below, where Φ⁻¹ is the inverse of the standard normal distribution function. The probabilities are estimated by Monte Carlo sampling, so certificates hold with high probability, and the base model is trained with noise augmentation. Unlike exact verification it scales to ImageNet-sized networks, but it needs many forward passes per prediction and certifies only modest radii.

    With σ = 0.25, the smoothed model certifies most test images at ℓ2 radius 0.25 but few beyond 1.0.

R=σ2(Φ−1(pA)−Φ−1(pB))R = \frac{\sigma}{2}\left(\Phi^{-1}(p_A) - \Phi^{-1}(p_B)\right)

Origin

Mathias Lecuyer and colleagues (PixelDP, 2018) derived noise-based certificates from differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person., and Bai Li and colleagues gave related bounds. Jeremy Cohen, Elan Rosenfeld and J. Zico Kolter proved the tight ℓ2 certificate in 'Certified Adversarial Robustness via Randomized Smoothing' (arXiv February 2019; ICML 2019), which made the method standard.

Related

See also
  • certified robustnesscertified robustness, n.: A mathematical guarantee — not just a test result — that no change to an input within a set limit can alter a model's answer.
  • adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up.
  • differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
  • adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.

Sources

  1. Cohen, J., Rosenfeld, E. & Kolter, J. Z. (2019). Certified Adversarial Robustness via Randomized Smoothing. ICML 2019. arXiv:1902.02918.

Mentioned in

certified robustnesscertified robustness, n.: A mathematical guarantee — not just a test result — that no change to an input within a set limit can alter a model's answer.