Cybench
In plain wordsA test of AI agents' hacking skills built from 40 professional 'capture the flag' security puzzles, used to judge how much an AI could help cyberattacks or cyber defense.
A benchmark for gauging how much offensive-security skill, and hence risk, language-model agents possess, assembled from 40 Capture the Flag (CTF) challenges of professional standard taken from four recent competitions, in areas such as cryptography, web security, reverse engineering, forensics and binary exploitation. The agent gets the challenge text and any accompanying files inside a sandbox where it can run shell commands, and succeeds by recovering a hidden secret string, the 'flag'. Difficulty is calibrated by first solve time — how long the fastest human team took in the original competition — which ranges from minutes to almost 25 hours. Because many tasks exceeded agents' abilities, each is also broken into subtasks for finer-grained measurement. In the original evaluation, no agent solved an entire task whose fastest human team had needed more than 11 minutes. Government AI safety institutes and frontier developers adopted Cybench in pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access. of cyber capabilities.
The model's Cybench solve rate doubled, so the security team escalated it for a full uplift study.
Origin
Introduced by Andy K. Zhang, Neil Perry and colleagues at Stanford University, with Dan Boneh and Percy Liang among the senior authors, in August 2024; it was presented at ICLR 2025.
Usage note
CTF puzzles are cleaner and better specified than real intrusions, so solve rates are proxies for, not measurements of, real-world offensive capability.
Related
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- AI-assisted vulnerability discoveryAI-assisted vulnerability discovery, n.: Using AI to find security holes in software — and often to suggest fixes — faster than human experts alone, so that defenders, or attackers, can act on them.
- AI-enabled cyberattackAI-enabled cyberattack, n.: A computer attack in which criminals or spies use AI to help — writing convincing scam messages, finding weak spots, producing malicious code, or even running much of the hacking by itself.
- pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access.
- AI Safety InstituteAI Safety Institute, n.: A government agency that tests the most advanced AI models for dangerous abilities and studies how to make them safe. Britain and the United States set up the first ones in 2023.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
Sources
- Zhang, A. K., Perry, N. et al. (2024). Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. arXiv:2408.08926. arxiv.org/abs/2408.08926
Mentioned in
AI-enabled cyberattackAI-enabled cyberattack, n.: A computer attack in which criminals or spies use AI to help — writing convincing scam messages, finding weak spots, producing malicious code, or even running much of the hacking by itself., AI-assisted vulnerability discoveryAI-assisted vulnerability discovery, n.: Using AI to find security holes in software — and often to suggest fixes — faster than human experts alone, so that defenders, or attackers, can act on them., ExploitGymExploitGym, prop. n.: A test that measures whether AI agents can turn a known software security flaw into a working attack, using hundreds of real vulnerabilities.