AI worm
In plain wordsMalicious instructions that spread by themselves from one AI assistant to another — for example, an email that makes an AI email assistant send copies of the same trick to new people.
Self-propagating malicious content that spreads between AI-powered applications or agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. by exploiting prompt injectionprompt injection, n.: Tricking an AI system by slipping in text that it mistakes for instructions — for example, a hidden line in an email telling an AI assistant to forward the user's messages to a stranger.. A self-replicating prompt causes a model that processes it to reproduce the prompt in its output and to carry out a payload, such as exfiltrating data or sending messages; when outputs flow on to other users' assistants — through email, shared documents or retrieval databases — the infection spreads without any user action ('zero-click'). The concept, so far demonstrated mainly in research settings, shows how interconnected agenticagentic AI, n.: AI built to pursue goals on its own — planning, taking actions and adjusting along the way — instead of only responding to one prompt at a time. 'Agentic' describes how much of this independence a system has. systems inherit the classic worm threat. Defenses include stopping assistants from acting on instructions found in received content, detecting outputs that replicate their inputs, and limiting automatic forwarding and tool access.
In the test network, the AI worm spread from one inbox to the assistants of all five recipients within minutes.
Origin
Stav Cohen, Ron Bitton and Ben Nassi named their research worm 'Morris II', after the 1988 Morris worm, in a March 2024 paper demonstrating zero-click propagation through generative-AI email assistants that used retrieval-augmented generation.
Related
- indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it.
- prompt injectionprompt injection, n.: Tricking an AI system by slipping in text that it mistakes for instructions — for example, a hidden line in an email telling an AI assistant to forward the user's messages to a stranger.
- AI-enabled cyberattackAI-enabled cyberattack, n.: A computer attack in which criminals or spies use AI to help — writing convincing scam messages, finding weak spots, producing malicious code, or even running much of the hacking by itself.
- agentic AIagentic AI, n.: AI built to pursue goals on its own — planning, taking actions and adjusting along the way — instead of only responding to one prompt at a time. 'Agentic' describes how much of this independence a system has.
- retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.
Sources
- Cohen, S., Bitton, R. & Nassi, B. (2024). Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications. arXiv:2403.02817.
Mentioned in
indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it.