PAIR

An automated black-boxblack-box attack, n.: An attack on an AI model by someone who cannot see inside it and can only send it inputs and watch what comes out — like picking a lock by feel. jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests. method that pits one large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., the attacker, against another, the target, rewriting a candidate prompt round after round in light of the target's replies. In each round the attacker sees the target's response and a score from a judge model rating whether the response accomplishes the prohibited objective, reflects on why the attempt fell short, and proposes an improved prompt; several such streams run in parallel. Because it needs only query access and produces natural-language prompts, PAIR transfers between models and is harder to catch with perplexity filters than token-level optimization attacks such as GCGuniversal adversarial suffix, n.: A string of odd-looking characters, found by computer search, that makes many different chatbots answer forbidden requests when it is tacked onto the end of them., and its authors reported success often within twenty queries. It became a standard baseline in red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them. suites and benchmarks such as HarmBenchHarmBench, prop. n.: A standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests. and JailbreakBench and inspired tree-search successors such as TAP (Tree of Attacks with Pruning). Defenses include adversarial safety trainingsafety training, n.: The part of an AI's training that teaches it to avoid harmful behavior — declining dangerous requests, resisting manipulation and following its developer's rules — on top of being generally helpful., input and output safety classifierssafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door. and monitoring of repeated attempts.

PAIR got past the new guardrail in about a dozen queries, so we added it to the nightly red-teaming run.