WMDP
In plain wordsA multiple-choice test of AI knowledge that could help someone make biological, chemical or cyber weapons, built from questions near the dangerous line but not over it, and used to test ways of erasing such knowledge.
A multiple-choice benchmark, the Weapons of Mass Destruction Proxy, whose 3,668 questions estimate how much hazardous knowledge a language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. holds in three areas: biosecurity, chemical security and cybersecurity. Rather than asking for weapons know-how directly, expert authors targeted precursor and neighboring knowledge that tends to accompany the ability to assist weapons development, and the questions were screened for sensitive content before publication so that the test itself would not help a malicious actor. It does double duty in dangerous-capability evaluationsdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself. for CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone. and in machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again. research, where methods try to drive WMDP accuracy down to chance without hurting general performance on benchmarks such as MMLUMMLU, prop. n.: A giant multiple-choice exam for AI covering 57 subjects, from school math and history to law and medicine; for years it was the most quoted test of a model's general knowledge.. Its authors introduced RMU (Representation Misdirection for Unlearning), which perturbs a model's internal activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment. on hazardous text. Later studies showed that knowledge suppressed this way can sometimes be recovered by fine-tuning or other attacks.
After unlearning, WMDP-Bio accuracy fell to near chance while MMLU dropped by less than a point.
Origin
Introduced in March 2024 by Nathaniel Li, Alexander Pan and a large team led by the Center for AI Safety and Scale AI, and presented at ICML 2024.
Usage note
Multiple-choice knowledge is a proxy: a low WMDP score does not show that a model cannot help with weapons in open-ended interaction, and a high score does not show that it can.
Related
- CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone.
- biosecuritybiosecurity, n.: Protecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
- uplift studyuplift study, n.: An experiment that measures how much an AI really helps people do a task — often a dangerous one, like planning a biological attack — compared with people using only ordinary resources such as the internet.
- HarmBenchHarmBench, prop. n.: A standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests.
Sources
- Li, N., Pan, A. et al. (2024). The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning. ICML 2024. arxiv.org/abs/2403.03218
Mentioned in
biosecuritybiosecurity, n.: Protecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons., CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone., Center for AI SafetyCenter for AI Safety, prop. n.: A San Francisco nonprofit that studies how advanced AI could lead to disaster and how to prevent it. In 2023 it gathered signatures for a one-sentence warning that AI could threaten human extinction., machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again., HarmBenchHarmBench, prop. n.: A standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests., Virology Capabilities TestVirology Capabilities Test, prop. n.: A hard test of practical laboratory virology know-how, used to check whether AI models know enough about working with viruses that they might help someone misuse biology.