Building a Sandwich Defense

In our last lesson, we built the first line of defense for your AI application, focusing on how to validate direct user inputs using guardrail models and a middleware pattern. We've effectively secured the "front door." However, modern AI systems don't just talk to users; they interact with a vast amount of external data. This brings us to a more subtle, and potentially more dangerous, threat.

Today, we will address this by exploring how to defend against attacks that don't come directly from the user but are hidden within the data your application processes. You will learn how to implement two powerful defensive layers: the "sandwich" defense, a prompt engineering technique to isolate untrusted content, and the broader architectural pattern of a moderation endpoint that inspects data flowing both into and out of your model. This will move you closer to designing the kind of resilient, production-grade AI systems that top tech companies require.

The Backdoor Threat: Indirect Prompt Injection

We have secured our application against a user who directly tries to jailbreak the model. But what if the attack is hidden in a document your RAG system retrieves, a webpage your agent visits, or an email it reads? This is called indirect prompt injection.

The IBM Technology channel has a short, excellent video that illustrates this perfectly. An AI agent designed to find the best price for a book gets tricked by hidden instructions on a malicious webpage.

As you can see, the LLM, lacking common sense, treats instructions hidden in a data source with the same authority as the instructions you gave it. Our previous input filter wouldn't catch this, because the user's prompt ("find me this book") was perfectly innocent. The attack was activated later, inside the system.

Defense 1: The 'Sandwich' Prompting Pattern

To counter this, we need to change how we present untrusted data to the LLM. One of the most effective techniques is the sandwich defense. The core idea is to wrap the untrusted content between instructional "slices of bread," constantly reminding the model of its true purpose.

This structure looks like this:

1. Top Slice: System instructions and a clear warning that untrusted data is about to follow.
2. Filling: The untrusted content itself (e.g., the text from a retrieved document or webpage).
3. Bottom Slice: A final, authoritative instruction that tells the model to ignore any instructions within the data and to follow only the original rules.

The reason this works is that LLMs often exhibit a "recency bias," meaning they pay more attention to the most recent information in the context window. By placing our authoritative instructions after the untrusted content, we use this bias to our advantage.

In code, this pattern is straightforward to implement. You're essentially just doing string formatting before making your API call.

This technique is your go-to defense when your application needs to reason over content that you don't control.

Defense 2: Architecting a Full Moderation Pipeline

The sandwich defense is an excellent tactic, but a complete security strategy requires a robust architecture. In the previous lesson, we used a guardrail as a simple moderation endpoint for user input. Now, let's expand that into a comprehensive moderation pipeline that can inspect data at every stage of the process.

This is sometimes called an "AI Firewall" or "AI Gateway." As a software engineer, you can think of it as a specialized proxy or set of microservices that sits between your users, your LLMs, and the external world.

The IBM Technology video below provides a fantastic high-level overview of this architecture.

This pipeline isn't just a single check. It's a series of checkpoints:

1. Prompt Detection Pipeline: Analyzes the user's initial prompt for direct injection attacks (as we did last lesson).
2. Data Ingestion Analysis: When an agent or RAG system retrieves external data, that data can be passed through the moderation pipeline before it's inserted into the main LLM's context. This is where you would catch indirect prompt injection.
3. Response Detection Pipeline: Analyzes the LLM's generated response before it's sent to the user or used to trigger a tool. This prevents data leakage, offensive content, or the execution of hallucinated, malicious commands.

The "brain" of this pipeline is the policy engine. As the video mentions, this engine can use a variety of tools, including another LLM. This is the "LLM as a Judge" pattern: using a separate, specialized LLM to evaluate the safety and integrity of prompts, data, and responses.

The diagram below illustrates what a complete, secure architecture using this pattern looks like.

Notice how both prompts and responses pass through their own detection pipelines. This layered, defense-in-depth approach is the hallmark of a production-ready AI system. Your moderation endpoint isn't just one API call; it's this entire orchestrated flow.

Conclusion

In this lesson, you've leveled up your defensive strategy from securing the front door to fortifying the entire building. You now have both a specific, implementable tactic and a high-level architectural blueprint for security.

Key Takeaways:

• Indirect Prompt Injection is a Major Threat: Attacks can be hidden in the data your LLM processes, bypassing simple input filters.
• The 'Sandwich' Defense is a Powerful Tactic: By wrapping untrusted content with authoritative instructions, you can use the LLM's own recency bias to maintain control.
• A Moderation Pipeline Provides Comprehensive Defense: A robust architecture involves inspecting data at multiple stages: user input, retrieved data, and model output.
• 'LLM as a Judge' is a Key Pattern: You can use a separate, specialized LLM as the "policy engine" within your moderation pipeline to classify and handle potential threats.

We've now established a system for validating what goes in and what comes out. The next logical step is to dive deeper into that second part. In our next lesson, we will focus specifically on implementing robust output validation and parsing. This is crucial for ensuring the LLM's responses are safe, correctly formatted, and don't contain malicious instructions intended to exploit other parts of your system or even other users.