Designing AI Agents That Survive Prompt Injection

Every week someone ships an agent that reads your email, your documents or a web page, and can also send things out on your behalf. That combination is the whole problem. The moment an agent reads something an attacker controls, the attacker gets a vote on what the agent does next.

We have a name for this now: the lethal trifecta . Private data, untrusted input, and a way to send data out. Put all three in one agent and a single poisoned email or web page can turn it against you. Simon Willison coined the phrase, and once you see it you cannot unsee it.

The uncomfortable part is that the usual fixes do not fix it.

Why “just filter it” does not work

The first instinct is always the same. Add a stronger system prompt. Tell the model to ignore instructions in the data. Put a guardrail model in front to catch the bad stuff.

These help a little. None of them are guarantees. In 2025, researchers ran adaptive attacks against a dozen published defences and got past most of them more than ninety percent of the time. The reason is simple. A model cannot reliably tell the difference between instructions and data, because to the model it is all just text in the same window. Any filter you bolt on is another model making the same kind of guess, and attackers are very good at finding the small percentage it misses.

In security, ninety five percent effective is not a win. The five percent is the whole game.

So if we cannot make the model immune, what do we do?

The one principle that changes everything

Here is the shift. Stop trying to make the model behave. Assume it will be fooled, every single time, and design the system around it so that being fooled does not matter.

The researchers behind the design patterns paper (Beurer-Kellner and colleagues) put it well. Once an agent has taken in untrusted input, it must be constrained so that input cannot trigger any consequential action. Not “less likely to.” Cannot.

That sounds limiting, and it is. You give up some of the agent’s ability to do absolutely anything you ask. In exchange you get something you can actually reason about. For most real systems that is a trade worth making, and it is the trade almost nobody is making today.

Once you accept that framing, a handful of design patterns fall out of it. None of them are mine. They come from Simon Willison’s early work and from the CaMeL paper out of Google DeepMind, pulled together in that 2025 design patterns paper. I just find them clearer when you see the thread connecting them.

The patterns, in plain terms

Let the model pick, not act. The simplest safe design lets the model choose from a fixed set of actions and nothing more. Think of it as a switch statement the model operates. It can decide “show the user this page” or “send this canned response,” but it never sees the result of an action and acts on it. No feedback loop, no injection path. You lose a lot of flexibility, and for a narrow task that is fine.

Decide the plan before you read the data. A step up: let the model plan the sequence of actions first, before it touches any untrusted content. The plan is frozen. An injection in the data can still mess with the contents of a step, but it cannot add new steps or change who an email goes to. You have protected the control flow even if you have not protected everything.

Handle each piece of poison in isolation. When you have to process a pile of untrusted things, like a folder of files or a batch of reviews, handle each one in its own isolated pass that cannot call dangerous tools, then combine the results with plain code. One malicious item can only affect its own little result, not the whole job.

Keep the privileged part blind. This is Willison’s dual LLM idea. One model plans and holds the tools but never sees untrusted text. A second, quarantined model reads the untrusted text but has no tools. Normal software sits in the middle and passes references around, never letting the tainted text reach the privileged model. The planner cannot be hijacked by something it never reads.

Turn the plan into checked code. CaMeL takes the dual LLM idea further. The privileged model writes a small program that describes exactly which tools get called and how data flows between them. Because the data flow is explicit, the system can tag where every value came from and refuse to, say, email a document to someone who was never supposed to see it. This is the most powerful of the patterns and also the most work to build.

Forget what you do not need. Finally, drop content from the context when it has done its job, especially the user’s own prompt. If a request has already been turned into a database query, you do not need the original wording hanging around to be abused later.

What this really means

Notice what these patterns have in common. Not one of them tries to make the model smarter or more obedient. They change the shape of the system so that a fooled model cannot reach anything that matters.

None of them is a silver bullet on its own, and the research is honest about that. The right answer is usually a combination, chosen for the specific agent you are building. The real lesson is a mindset. Build agents that do one job inside clear boundaries, instead of general-purpose agents that can do anything and therefore can be made to do anything.

If you are building with LLMs right now, that is the question worth asking before you ship. Not “how do I stop the model from being tricked,” but “when it is tricked, what is the worst it can actually do?” Answer that honestly, and the design mostly writes itself.

Further reading

• Simon Willison, The Dual LLM pattern and The lethal trifecta for AI agents
• Debenedetti et al., Defeating Prompt Injections by Design (CaMeL)
• Beurer-Kellner et al., Design Patterns for Securing LLM Agents against Prompt Injections
• Nasr et al., The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections