AgentDojo

An extensible benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time. and test harness measuring how well AI agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. resist indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it. when the tools they call return content that an attacker may control. It supplies simulated environments — a workspace with email, calendar and cloud files, a Slack-style messaging workspace, travel booking and e-banking — with realistic user tasks and attacker goals; the original release had 97 user tasks and 629 security test cases. Injected instructions are placed in tooltool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on. outputs such as emails, reviews or web pages, and the framework measures both utility (whether the agent completes the user's task) and attack success (whether it carries out the attacker's goal), with and without defenses such as prompt sandwiching, injection detectors and tool filtering. Because success is checked by inspecting the environment's state rather than the agent's text, new tasks, attacks and defenses are easy to add. It became a standard testbed for agent hijacking, used by the US AI Safety Institute and in industry and academic research on defenses.

The new injection detector cut AgentDojo attack success to 3%, but utility dropped eight points.