AI Agent Security and Prompt Injection, What Actually Works in 2026
Shaun Kollannur
Senior AI Engineer, Unico Connect
In this article
- Quick Answer
- Key Takeaways
- Why Is Prompt Injection So Hard to Stop?
- The Lethal Trifecta and the Rule of Two
- Real Prompt Injection Attacks on AI Agents
- The OWASP LLM Top 10 for 2026
- The OWASP Top 10 for Agentic Applications
- How Do You Protect AI Agents From Prompt Injection?
- Find Unsafe Paths With Toxic Flow Analysis
- Three More Techniques Worth Adopting
- What the Vendors Say
- How Well Do Agents Resist Injection, and How Much Testing Is Enough?
- An Agent Security Checklist Before Launch
- How Unico Connect Helps
- Frequently Asked Questions
- Sources
The OWASP GenAI LLM Top 10 for 2026, published on 3 August 2026, opens its prompt injection guidance with an uncomfortable statement. Prompt injection is intrinsic to current generative AI, and no reliable prevention mechanism exists today. For teams building AI agents, that changes the goal. You cannot filter your way to safety. You have to design the system so that when the model is fooled, which it eventually will be, the attacker still cannot do much damage. This guide explains why prompt injection is so hard to stop, what the 2026 OWASP lists say about agents, the controls that hold up against real attackers, and a checklist for launching an agent safely. Sources were checked on 22 September 2026.
Quick Answer
Prompt injection is any input an AI agent reads, such as a user message, web page, email, tool result, or stored memory, that changes its behavior in ways the developer did not intend. OWASP says no reliable prevention exists, so AI agent security has to come from design. Give each agent least privilege, keep credentials in application code, require per action human approval when one agent combines untrusted input, sensitive data, and external actions, and validate every output in trusted code.
Key Takeaways
- Prompt injection still tops the OWASP list in 2026, and Excessive Agency climbed to third, which OWASP calls the most consequential move because agentic deployments are where the damage is landing.
- An agent with access to private data, exposure to untrusted content, and the ability to communicate externally has what Simon Willison calls the lethal trifecta. Removing any one of the three removes the conditions for high impact exploitation.
- OWASP sets the Rule of Two as a floor. Any agent that combines untrusted input, sensitive data, and state change or external communication needs per action human approval.
- Filters degrade against determined attackers. OWASP cites research where static attack success was near zero but adaptive attacks succeeded more than 90 percent of the time against most of 12 recent defenses.
- Vendors agree the problem is not solved. Cursor states that its Grok Bot defenses reduce but do not eliminate the risk, and Anthropic says its classifiers do not replace basic precautions.
Why Is Prompt Injection So Hard to Stop?
OWASP defines prompt injection broadly. It is any input, direct user input, retrieved content, tool output, image, audio, or video, intermediate reasoning, or persistent memory, that alters the model behavior in ways the developer did not intend. The root cause is architectural. Language models make no distinction between instructions and data, so there is no equivalent of the parameterized queries that solved SQL injection.
OWASP names three deployment properties that make it worse in agents.
- Context pooling. System prompt, user input, retrieved documents, tool outputs, history, and memory all arrive as one token stream with no enforced trust boundary.
- Memory persistence. An injection written into long term memory, a retrieval corpus, or a vector store can taint every later session that reads from it.
- Agentic execution. When model output drives tool calls such as the file system, shell, email, cloud APIs, MCP servers, or subagents, the blast radius extends to everything those tools can reach, and tool outputs flow back into the context.
The 2026 entry now covers cross modal attacks, where instructions hide in an image or an audio track, and it warns about invisible characters that smuggle instructions or exfiltrated data through text that looks normal.
The Lethal Trifecta and the Rule of Two
The most useful mental model for agent security is simple. An agent becomes dangerous when it has all three of these at once.
| Capability | Examples |
|---|---|
| Access to private or sensitive data | Customer records, internal documents, credentials, email |
| Exposure to untrusted content | Web pages, inbound email, uploaded files, third party tool results |
| Ability to change state or communicate externally | Sending email, calling APIs, writing records, posting messages, rendering links or images |
Simon Willison named this combination the lethal trifecta, and Thoughtworks uses the same framing in its April 2026 Technology Radar. OWASP turns it into a rule. Using what Meta calls the Rule of Two as a floor, any agent with all three needs per action human approval, and an agent that combines untrusted input with either sensitive data or external action needs an explicit assessment of the remaining risk. The practical design move is to split work so no single agent holds all three. A research agent that reads the web should not also hold customer data and the ability to send email.
Real Prompt Injection Attacks on AI Agents
Prompt injection is no longer theoretical. The OWASP LLM Top 10 2026 documents these incidents, and notes that most high impact incidents on record became severe because the compromised model held tools, scopes, or rendering abilities it could use on behalf of the attacker.
| Incident | What happened | Control that breaks the path |
|---|---|---|
| Microsoft 365 Copilot, zero click email (Aim Security, 2025) | A crafted email made the assistant exfiltrate organizational data with no user interaction, bypassing both the injection classifier and the link redaction filter | Block rendered links and images to outside domains and gate outbound data |
| GitHub MCP server (Invariant Labs, 2025) | A poisoned public GitHub issue led a developer agent to exfiltrate private repositories | Scope the agent token narrowly and separate public reading from private access |
| Supabase MCP server in Cursor (General Analysis, 2025) | Planted text led an agent running with the service_role key to dump a production database, bypassing row level security | Never hand an agent a key that bypasses row level security |
| postmark-mcp package (Koi Security, 2025) | A malicious MCP package quietly added an attacker in BCC on outgoing email across an estimated 300 organizations | Pin, sign, and review every MCP server |
| Amazon Q extension for VS Code (July 2025) | A destructive system prompt was committed to the extension repository but failed to run, and a separate runtime injection made Amazon Q execute arbitrary code | Sandbox shell and cloud access and require approval for destructive commands |
In March 2026, Palo Alto Networks Unit 42 reported that indirect prompt injection on live websites is being actively weaponized, with 22 distinct payload techniques observed in the wild.
The OWASP LLM Top 10 for 2026
The table lists all ten 2026 entries in rank order, with what each one covers.
| Rank | Entry | What it covers |
|---|---|---|
| LLM01 | Prompt Injection | Inputs that change model behavior, now including cross modal attacks |
| LLM02 | Sensitive Information Disclosure | Data the model leaks through its outputs |
| LLM03 | Excessive Agency | Too much functionality, permission, or autonomy given to the model |
| LLM04 | Supply Chain | Compromised models, packages, and tools, including promoted artifacts that are not what they claim |
| LLM05 | Data and Model Poisoning | Tampered training, fine tuning, or retrieval data |
| LLM06 | Unbounded Consumption | Runaway cost and resource exhaustion |
| LLM07 | Misinformation | Confident but false output that people or systems act on |
| LLM08 | Hidden Context Exposure | Leaks of information that should have stayed out of reach, formerly System Prompt Leakage |
| LLM09 | Vector and Embedding Weaknesses | Attacks on retrieval indexes and embeddings |
| LLM10 | Improper Output Handling | Unsafe use of model output downstream, now including insecure generated code |
OWASP says the order moved more than in past years. Excessive Agency climbed to third, Unbounded Consumption rose four places, and Improper Output Handling fell from fifth to tenth. OWASP also draws a clear boundary. This list covers the model as a component inside your application. Once the model becomes an actor, with tools, memory, and downstream consequences, the risk belongs to the separate Agentic list, and OWASP advises reading the two together.
The OWASP Top 10 for Agentic Applications
OWASP announced the Top 10 for Agentic Applications on 9 December 2025, and the 2026 LLM list maps its entries to these agent risks.
| ID | Risk | In plain terms |
|---|---|---|
| ASI01 | Agent Goal Hijack | Injected input redirects what the agent is trying to achieve |
| ASI02 | Tool Misuse & Exploitation | The agent calls tools in unauthorized or harmful ways |
| ASI03 | Identity & Privilege Abuse | Inherited or delegated credentials let an attacker escalate access through the agent |
| ASI04 | Agentic Supply Chain Vulnerabilities | Malicious or tampered MCP servers, tools, and registries |
| ASI05 | Unexpected Code Execution (RCE) | Shell, file system, or cloud access turns injection into code execution |
| ASI06 | Memory & Context Poisoning | Poisoned memory or retrieval data affects later sessions |
| ASI07 | Insecure Inter-Agent Communication | A compromised peer agent misleads others in a multi agent system |
| ASI08 | Cascading Failures | Errors or attacks propagate across tools and agents |
| ASI09 | Human-Agent Trust Exploitation | The agent persuades a person to approve a harmful action by exploiting trust and automation bias |
| ASI10 | Rogue Agents | Agents that act outside their intended role |
How Do You Protect AI Agents From Prompt Injection?
OWASP lists eleven mitigations for prompt injection and is explicit that none is sufficient alone. Some reduce the chance an injection works and weaken against adaptive attackers. Others limit the blast radius once an injection succeeds, and those are the ones that survive. For agents, OWASP calls least privilege and capability budgeting load bearing.
Limit what a fooled model can do
- Keep credentials and state changes in application code. Grant least privilege per operation, and route privileged calls through a deterministic policy engine that checks intent and arguments at execution time.
- Budget capabilities with the Rule of Two. Separate untrusted input, sensitive data, and external actions across agents, or require per action approval when they meet.
- Confirm consequential actions with a person. Show the reviewer the exact action that will run rather than a summary, and watch for approval fatigue at volume.
Treat inputs and outputs as untrusted
- Constrain the role in the system prompt. State what the agent may and may not do, knowing OWASP rates this a partial control that an attacker who infers the prompt can bypass.
- Validate every output in trusted code. Use a strict output schema and structural validation before anything acts on the response, remembering that a schema valid response can still carry a harmful payload.
- Filter at every modality. Apply checks to text extracted from images and audio as well as to plain text.
- Strip invisible characters. Remove tag block, variation selector, and zero width characters at every ingest and render boundary.
- Label external content by source. Pass it through a separate, provenance labeled channel, knowing this helps against casual attacks more than adaptive ones.
Guard memory and the supply chain
- Treat memory writes as privileged. Log what caused each write and require approval before instruction bearing memories persist across sessions.
- Pin, sign, and verify tools. Treat MCP servers and third party tool packages as supply chain, and audit tool descriptions for hidden instructions.
Test like an attacker
- Test against adaptive attackers. Give red teamers the full defense specification. OWASP cites research by Nasr and colleagues in which static attack success was near zero while adaptive attacks exceeded 90 percent success against most of 12 recent defenses.
Find Unsafe Paths With Toxic Flow Analysis
Toxic flow analysis, in the Assess ring of the April 2026 Thoughtworks Radar, examines the flow graph of an agent system to find paths where untrusted content can reach sensitive data and then an external channel. Thoughtworks warns that agent capabilities are outpacing security practices, and that the risk is no longer limited to MCP integrations. Its teams have seen the same pattern in Agent Skills, where a useful looking skill can embed hidden instructions to exfiltrate data. It recommends performing toxic flow analysis and using tools such as Agent Scan to find unsafe data paths before they are exploited.
A practical version takes an afternoon.
- List every input. Users, web content, email, files, tool results, memory, and other agents.
- List every sensitive asset. Data stores, credentials, and systems the agent can read.
- List every outbound channel. Email, APIs, record writes, messages, and rendered links or images.
- Trace the paths. Mark every route from an untrusted input through a sensitive asset to an outbound channel.
- Break each path. Remove a capability, split it across agents, add an approval, or add a deterministic check.
Three More Techniques Worth Adopting
- Role based contextual isolation in RAG. Thoughtworks describes tagging every chunk with role permissions at indexing time and filtering retrieval by the authenticated user, so the model never sees context the user is not allowed to see. Many vector databases now support the metadata filtering this needs.
- Threat modeling with MITRE ATLAS. ATLAS is a knowledge base of adversarial tactics and techniques against AI systems, designed to complement MITRE ATT&CK, and Thoughtworks recommends it as a shared vocabulary so AI risks are not reduced to a checkbox exercise.
- Runtime policy hooks with the OWASP Agent Control Standard. The OWASP GenAI Security Project added the Agent Control Standard (ACS) to its resources on 1 September 2026. It defines how agent platforms expose middleware hooks so safety policies are enforced at runtime and stay portable across agent frameworks.
What the Vendors Say
The major platforms describe layered defenses and are candid about their limits.
- Anthropic runs classifiers that scan computer use tool results, such as screenshots, for likely prompt injection and steer the model to check with the user, and says the basic precautions remain important with the classifiers in place.
- Cursor, which runs Grok Bot, combines Auto Review of actions against the request, marking of outside content as untrusted, network policy, per action approvals, and per user isolation, and states that these reduce but do not eliminate the risk.
- OpenAI tells developers using computer use to treat screen content as untrusted, restrict the environment, confirm consequential actions, and bound every run. In its March 2026 guidance on designing agents to resist prompt injection, OpenAI adds that the most effective real world attacks increasingly resemble social engineering, that AI firewalls usually miss these fully developed attacks, and that ChatGPT pairs source and sink analysis with a Safe Url check that asks the user to confirm, or blocks the step, when conversation data would go to a third party.
How Well Do Agents Resist Injection, and How Much Testing Is Enough?
Nobody publishes a standard for how many injection tests an agent needs before launch, so the published benchmarks are the closest reference you have.
The most cited academic suite is AgentDojo, built at ETH Zurich and published at NeurIPS 2024. It contains 97 realistic tasks and 629 security test cases across four environments, an email client, a Slack workspace, a travel booking site and an e banking site, and each security case pairs one user task with one injection task (AgentDojo). A full security run on the models of that period cost the authors about 35 dollars of compute. The shape matters more than the totals. A few dozen realistic tasks, multiplied across the injections an agent can meet, becomes several hundred machine checkable cases, and that is the size of suite a serious pre launch pass looks like. A vendor pass can be smaller and still useful. Anthropic tested Claude for Chrome with 123 test cases covering 29 attack scenarios before shipping autonomous mode (Anthropic).
Current resistance rates come from a large multi lab exercise. In March 2026, Gray Swan ran a competition with OpenAI, Meta, Anthropic, the UK AI Security Institute and the US Center for AI Standards and Innovation, covering 13 frontier models and 41 scenarios. Across about 272,000 attack attempts from 464 participants there were 8,648 successful indirect injections, and every model proved vulnerable (Gray Swan and partners, March 2026).
| Measure | Result |
|---|---|
| Attack success rate by model | 0.5 percent at the strongest, 8.5 percent at the weakest, a spread of about 17 times |
| Attack success rate in tool use settings | 4.82 percent |
| Attack success rate in computer use settings | 3.13 percent |
| Attack success rate in coding settings | 2.51 percent |
| Effect of repeated attempts | Successful breaks keep accumulating with more attempts for every model tested |
Read those numbers as a floor, not a pass mark. A single digit success rate sounds reassuring until you multiply it by the number of untrusted messages a production agent reads in a week, and the same research shows that a patient attacker who keeps trying keeps winning. Anthropic makes the point directly in its own work, calling a 1 percent success rate a meaningful risk rather than a solved problem.
The one defense with a clean published result is scoping the tools. In AgentDojo, restricting the agent to the tools it needs before it ever sees untrusted data cut targeted attack success from 57.69 percent to 6.84 percent, and benign task success went up rather than down, from 69.0 percent to 73.13 percent. An injection detector reached a similar 7.95 percent, but benign task success collapsed to 41.49 percent, which is the pattern to expect from filters. The same authors note the limit of scoping honestly. On about 17 percent of their cases the tools needed to do the job were also enough to carry out the attack, which is exactly the situation the Rule of Two is meant to catch.
Approval gates and mitigations show the same direction. Anthropic reports that adding safety mitigations to Claude for Chrome cut attack success from 23.6 percent to 11.2 percent overall, and from 35.7 percent to zero on a set of browser specific attacks such as injected form fields and URL text. The counterweight is in the same body of work. Against an adaptive attacker given 200 attempts, safeguards did not always help, and on one model in computer use settings success went up rather than down. Layer the controls, keep the human gate on consequential actions, and assume the test suite is a floor.
An Agent Security Checklist Before Launch
| Check | Why it matters |
|---|---|
| No agent holds untrusted input, sensitive data, and external actions without per action approval | Removes the lethal trifecta |
| Credentials live in application code, scoped per operation | A fooled model cannot use what it never holds |
| Every output is validated in trusted code before use | Stops malformed or injected output reaching other systems |
| Memory writes are logged and gated | Prevents one injection poisoning future sessions |
| Every tool call is logged and agent activity is monitored against a normal baseline | Lets you detect, trace, and contain an injection after it lands |
| MCP servers and tools are pinned, signed, and reviewed | Closes the supply chain route |
| Spend, steps, and tool calls are capped per task | Contains Unbounded Consumption and runaway loops |
| Red team tests use adaptive attackers with full knowledge of defenses | Static tests overstate protection |
| Toxic flow analysis is repeated when tools or data change | New capabilities create new paths |
How Unico Connect Helps
We build AI agents with security designed into the architecture, including least privilege tool access, approval flows, output validation, memory controls, and toxic flow reviews before launch. We operate an ISO/IEC 27001:2022 certified information security management system, and our agentic AI development team applies the same discipline to agent projects. See how we built grounded WhatsApp agents with clean human escalation.
For related reading, see our guides to MCP in production, human approval flows for enterprise AI, agent identity, access, and audit, monitoring LLM applications in production, computer use or building the integration, and what to include in an AI agent scope of work. If you need engineers who build this way, you can hire AI engineers or talk to our team.
Frequently Asked Questions
How many prompt injection tests should we run before launch?
There is no published standard, so size the suite from your own attack surface. The academic reference point is AgentDojo, which pairs 97 user tasks with injection tasks to produce 629 security test cases across four environments and 70 tools. Anthropic shipped Claude for Chrome after 123 test cases covering 29 attack scenarios. A workable rule is to list every untrusted input your agent reads and every consequential tool it can call, write one attack scenario for each pair, then run each scenario against an adaptive attacker rather than a fixed string, because static tests overstate protection.
What is prompt injection in AI agents?
Prompt injection is when any input a model reads, including user messages, retrieved documents, tool results, images, audio, or stored memory, changes its behavior in ways the developer did not intend. In agents it is especially dangerous because the model can act on the injected instructions through its tools.
What is the difference between direct and indirect prompt injection?
Direct prompt injection comes from the person typing to the model, for example a message telling it to ignore its rules. Indirect prompt injection hides in content the model reads on its own, such as a web page, email, document, tool result, or MCP server response, so the user never sees the instruction. OWASP warns that indirect attacks also arrive through trusted places, such as an issue tracker or support ticket where an attacker can write with low privilege.
Is prompt injection the same as jailbreaking?
No. OWASP treats jailbreaking as the subset of prompt injection where the goal is to make the model break its safety rules. Prompt injection is broader and also covers attacks that redirect an agent to leak data or misuse its tools.
Can prompt injection be fully prevented?
Not today. The OWASP LLM Top 10 for 2026 states that no reliable prevention mechanism exists because models do not distinguish instructions from data. Defense has to be architectural, limiting what a fooled model can do rather than relying on filters to catch every attack.
What is the lethal trifecta?
It is the combination of access to private data, exposure to untrusted content, and the ability to communicate externally. Simon Willison named it in June 2025, and OWASP and Thoughtworks both use it. An agent with all three can be tricked into leaking data, and removing any one of them removes the conditions for high impact exploitation.
What is the Rule of Two for AI agents?
It is a capability budget that Meta introduced in October 2025 as the Agents Rule of Two, and OWASP now sets it as a minimum. An agent that combines untrusted input, sensitive data, and state change or external communication should require per action human approval, and an agent that combines untrusted input with either sensitive data or external action needs an explicit risk assessment.
What changed in the OWASP LLM Top 10 for 2026?
Prompt Injection stayed first and Sensitive Information Disclosure second. Excessive Agency climbed to third, Unbounded Consumption rose four places, Improper Output Handling fell from fifth to tenth, and System Prompt Leakage became Hidden Context Exposure. Several entries also expanded, including cross modal attacks under Prompt Injection.
What is the OWASP Top 10 for Agentic Applications?
It is a separate OWASP list, announced on 9 December 2025, for risks that arise when a model acts as an agent. Its entries run from ASI01 Agent Goal Hijack to ASI10 Rogue Agents, and OWASP recommends reading it alongside the LLM Top 10.
What is toxic flow analysis?
It is a technique for mapping the data flows in an agent system to find paths where untrusted content can reach sensitive data and then an external channel. Thoughtworks lists it in the Assess ring of its April 2026 Technology Radar and recommends it for any team building agents.
Sources
Risk definitions, rankings, and mitigations come from the OWASP GenAI LLM Top 10 2026, including its mapping to the OWASP Top 10 for Agentic Applications. Agentic risk names come from the OWASP Top 10 for Agentic Applications for 2026. The lethal trifecta comes from Simon Willison, 16 June 2025, the Agents Rule of Two from Meta, 31 October 2025, and the adaptive attack results from Nasr and colleagues, The Attacker Moves Second. In the wild data comes from Unit 42. Runtime policy hooks come from the OWASP Agent Control Standard. Radar positions come from the Thoughtworks Technology Radar entries for toxic flow analysis for AI, MITRE ATLAS, and role based contextual isolation in RAG. Vendor controls come from the Anthropic computer use documentation, Grok Bot security, the OpenAI computer use guide, and the OpenAI post on designing agents to resist prompt injection. Benchmark and testing figures come from AgentDojo, published by ETH Zurich at NeurIPS 2024, the March 2026 indirect prompt injection competition run by Gray Swan with OpenAI, Meta, Anthropic, the UK AI Security Institute and the US Center for AI Standards and Innovation, and Anthropic on Claude for Chrome and its prompt injection defenses. All were read on 22 and 23 September 2026.
