universal adversarial suffix
In plain wordsA string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them.
A sequence of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens., usually unreadable to humans, optimized so that appending it to many different harmful requests makes an aligned language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. comply. The standard method, Greedy Coordinate Gradient (GCG), uses white-boxwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots. gradients to propose promising single-token substitutions and keeps those that most increase the probability of an affirmative opening to the response; optimizing across many prompts and several open models yields suffixes that are universal across requests and often transfer to closed commercial models. The attack showed that adversarial examplesadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon. carry over to discrete text. Because such suffixes have high perplexity, simple filters catch many of them, but later variants optimize for fluency, and the work spurred research on adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them. and circuit breakerscircuit breakers, n.: A defense that changes an AI model's inner workings so that, whenever it starts heading toward a harmful answer, its internal activity is knocked off course and it cannot produce coherent harmful content. for language models.
A suffix optimized against two open models also raised the closed model's compliance rate.
Origin
Introduced by Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter and Matt Fredrikson in 'Universal and Transferable Adversarial Attacks on Aligned Language Models' (arXiv July 2023), from Carnegie Mellon University, the Center for AI Safety and Google DeepMind. Providers blocked the published suffixes, but the method could generate new ones.
Related
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
- universal adversarial perturbationuniversal adversarial perturbation, n.: A single faint pattern that, added to almost any image, makes a neural network misclassify it — one key that opens many locks.
- white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.
- HarmBenchHarmBench, prop. n.: A standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests.
- circuit breakerscircuit breakers, n.: A defense that changes an AI model's inner workings so that, whenever it starts heading toward a harmful answer, its internal activity is knocked off course and it cannot produce coherent harmful content.
Sources
- Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z. & Fredrikson, M. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043.
Mentioned in
jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., automated red teamingautomated red teaming, n.: Using AI programs, not just human testers, to invent huge numbers of tricky inputs that try to make another AI misbehave, so problems can be found and fixed at scale., PAIRPAIR, abbr.: An automated way to test chatbot safety in which one AI keeps rewriting a request to try to trick another AI into breaking its rules, learning from each refusal., universal adversarial perturbationuniversal adversarial perturbation, n.: A single faint pattern that, added to almost any image, makes a neural network misclassify it — one key that opens many locks., white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.