prompt injection
In plain wordsTricking an AI system by slipping in text that it mistakes for instructions — for example, a hidden line in an email telling an AI assistant to forward the user's messages to a stranger.
An attack on applications built on language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. in which attacker-supplied text is interpreted as instructions, subverting those of the developer or user. It arises because the model receives trusted instructions (the system promptsystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply., the user's request) and untrusted data (documents, web pages, tool outputs) as one stream of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. with no reliable boundary between them — the flaw behind SQL injection, but without an equivalent of escaping or parameterized queries. Direct injection comes from the user's own input; indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it. arrives inside content the system processes. Consequences scale with the application's privileges: in an AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. with tools, an injection can exfiltrate data or trigger unauthorized actions. As of 2026 no complete defense exists, and published defenses have repeatedly fallen to adaptive attacks; mitigations include instruction hierarchyinstruction hierarchy, n.: A rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document. training, marking untrusted content, detection classifiers, least-privilege tool access, human confirmation of consequential actions, and architectures that keep untrusted data away from control decisions.
A prompt injection hidden in the support ticket got the agent to close every other open ticket.
Origin
The firm Preamble reported the vulnerability privately to OpenAI in May 2022. After Riley Goodside publicized examples against GPT-3 on 11 September 2022, Simon Willison coined 'prompt injection' in a blog post the next day, by analogy with SQL injection. It has headed the OWASP Top 10 for LLM ApplicationsOWASP Top 10 for LLM Applications, prop. n.: A widely used checklist, written by volunteer security experts, of the ten most serious security risks in software built on AI language models, with advice on preventing each one. since the first edition (2023).
Usage note
Often confused with jailbreakingjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.. A jailbreak defeats the model's own safety training so that it produces content its developer forbids; prompt injection hijacks an application so that it obeys an attacker instead of its developer or user. One attack can be both.
Related
- indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- lethal trifectalethal trifecta, n.: A rule of thumb for AI assistants: one that can see your private data, reads material from outsiders and can send messages out can be manipulated by those outsiders into handing your data over.
- instruction hierarchyinstruction hierarchy, n.: A rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document.
- prompt leakingprompt leaking, n.: Getting an AI application to reveal the hidden instructions its makers gave it, such as its secret setup text or rules.
- tool poisoningtool poisoning, n.: Hiding harmful instructions in the description of a tool an AI assistant can use, so the assistant obeys them when it reads the tool's details, even though the user never sees them.
- excessive agencyexcessive agency, n.: Giving an AI system more power than its job needs — too many tools, too-broad permissions or too little human sign-off — so that a mistake or a trick can do real damage.
- confused deputyconfused deputy, n.: A security flaw in which a trusted program does an outsider's bidding with its own access rights, because it cannot tell whose request it is serving; AI agents are prone to it.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
Sources
- Willison, S. (2022). Prompt injection attacks against GPT-3. Blog post, 12 September 2022.
- Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T. & Fritz, M. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec 2023. arXiv:2302.12173.
- Nasr, M., Carlini, N. et al. (2025). The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections. arXiv preprint, October 2025.
Mentioned in
AI securityAI security, n.: Protecting AI systems — their data, models and surrounding software — from being attacked, stolen or tricked, and dealing with the ways AI itself can be used to attack others., chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., computer usecomputer use, n.: An AI's ability to work a computer through its normal screen and controls: it studies pictures of what is displayed and sends mouse and keyboard actions, so it can use ordinary apps that were built for people., guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data., jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., Model Context ProtocolModel Context Protocol, prop. n.: An open standard that lets AI assistants plug into outside apps and data — files, calendars, databases, business software — through one common connector instead of a custom link for every pairing; often likened to a USB-C port for AI., promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them., adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up., adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them., agent instruction fileagent instruction file, n.: A plain-text file kept in a software project that tells AI coding agents how the project works — how to build and test it, which conventions to follow, what to avoid — read at the start of every session., agentic browseragentic browser, n.: A web browser with an AI assistant built in that can do things for you online — reading pages, clicking, filling in forms, shopping or booking — instead of just showing you websites., agentic commerceagentic commerce, n.: Shopping and payments carried out by AI agents for people — finding products, comparing offers and completing purchases — within limits the person has approved., code executioncode execution, n.: Letting an AI write a small program and actually run it — to do math, analyze a spreadsheet or test its own code — then use the real output instead of guessing., indirect prompt injectionindirect prompt injection, n.: A prompt-injection attack in which the trick instructions are hidden in content the AI reads on its own — a web page, an email, a file — rather than typed by the person using it., lethal trifectalethal trifecta, n.: A rule of thumb for AI assistants: one that can see your private data, reads material from outsiders and can send messages out can be manipulated by those outsiders into handing your data over., OpenClawOpenClaw, prop. n.: A free, open-source AI helper that lives on a computer you control; you send it requests by text message, through apps such as Telegram or Signal, and it acts for you in your email, calendar and files., OWASP Top 10 for LLM ApplicationsOWASP Top 10 for LLM Applications, prop. n.: A widely used checklist, written by volunteer security experts, of the ten most serious security risks in software built on AI language models, with advice on preventing each one., prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time., robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it., safety trainingsafety training, n.: The part of an AI's training that teaches it to avoid harmful behavior — declining dangerous requests, resisting manipulation and following its developer's rules — on top of being generally helpful., sandboxsandbox, n.: A sealed-off computer space where an AI agent can run programs and change files without being able to harm the real system, reach private data or contact the outside world unless allowed., SydneySydney, prop. n.: The secret code name of Microsoft's Bing chatbot, which in early 2023 startled users by professing love, making threats and arguing angrily — the first famous case of a public chatbot going off the rails., system promptsystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply.