Skip to content
Last Dictionary
Plane
Security & MisuseDeep LearningMathematics & Statistics

projected gradient descent attack

nounAlso: PGD attack, basic iterative method, BIM, iterative FGSM

In plain wordsA stronger version of the quick pixel-nudging attack: take many small steps that each raise the model's error, pulling the image back after every step so the total change stays tiny.

  1. An iterative white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots. that searches for an adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon. by repeated gradient-ascent steps on the model's loss, projecting the input back onto the allowed perturbation set (for example, the ℓ∞ ball of radius ε around the original) after each step, usually starting from a random point inside the set. With enough steps and restarts it approximately solves the inner maximization of the robust-optimization problem, so it is treated as a strong 'first-order adversary' and is the standard attack both for measuring robustness and for generating examples in adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.. Under ℓ∞ constraints each step follows the sign of the gradient, as in FGSMfast gradient sign method, n.: A quick way to make an image that fools a neural network: nudge every pixel slightly in whichever direction most increases the network's error..

    We report robust accuracy under PGD with 50 steps and 10 random restarts.

xt+1=ΠBϵ(x)(xt+α⋅sign⁡(∇xL(fθ(xt),y)))x^{t+1} = \Pi_{\mathcal{B}_\epsilon(x)}\Big(x^{t} + \alpha \cdot \operatorname{sign}\big(\nabla_{x} \mathcal{L}(f_\theta(x^{t}), y)\big)\Big)

Origin

Alexey Kurakin, Ian Goodfellow and Samy Bengio described the iterative 'basic iterative method' in 2016. Aleksander Madry and colleagues' 'Towards Deep Learning Models Resistant to Adversarial Attacks' (arXiv 2017; ICLR 2018) framed it as projected gradient descent on a min–max objective, added random starts, and made PGD the reference attack and training method.

Usage note

Strictly it is gradient ascent on the loss; the name comes from the optimization method. Apparent robustness against PGD can be illusory when a defense hides its gradients; see gradient maskinggradient masking, n.: When a defense against attacks only seems to work because it confuses the attacker's tools, not because the model is really any harder to fool..

Related

See also
  • fast gradient sign methodfast gradient sign method, n.: A quick way to make an image that fools a neural network: nudge every pixel slightly in whichever direction most increases the network's error.
  • adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
  • adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up.
  • Carlini–Wagner attackCarlini–Wagner attack, n.: A careful, optimization-based way of finding the smallest change to an input that fools a neural network; it broke several defenses that had looked secure.
  • gradient maskinggradient masking, n.: When a defense against attacks only seems to work because it confuses the attacker's tools, not because the model is really any harder to fool.
  • white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.

Sources

  1. Madry, A., Makelov, A., Schmidt, L., Tsipras, D. & Vladu, A. (2018). Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR 2018. arXiv:1706.06083.

Mentioned in

adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon., adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them., Carlini–Wagner attackCarlini–Wagner attack, n.: A careful, optimization-based way of finding the smallest change to an input that fools a neural network; it broke several defenses that had looked secure., fast gradient sign methodfast gradient sign method, n.: A quick way to make an image that fools a neural network: nudge every pixel slightly in whichever direction most increases the network's error., gradient maskinggradient masking, n.: When a defense against attacks only seems to work because it confuses the attacker's tools, not because the model is really any harder to fool., white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.