RAG Attack Surfaces & Indirect Injection

Hello! Welcome to the final lesson in our first module on LLM security foundations.

In our previous lesson, we explored how an attacker can exploit In-Context Learning (ICL) to perform direct prompt injection. By embedding malicious instructions or fabricated examples directly within the user prompt, an attacker can hijack the model's behavior for a single interaction. This is what the Deconvolute Labs article we'll see today calls a "Front Door" attack—it's synchronous, transient, and requires direct interaction.

Today, we shift our focus to a more persistent and scalable threat that arises when we connect LLMs to external knowledge. You will learn to diagram a Retrieval-Augmented Generation (RAG) pipeline and identify the new attack surfaces it introduces, particularly for indirect injection.

This is a critical foundation for your demo, as the vulnerable chatbot you will build is a RAG application. Understanding its architecture and weaknesses is the first step toward both exploiting and defending it.

1. What is Retrieval-Augmented Generation (RAG)?

LLMs have a "knowledge cutoff"—they don't know anything about events that occurred after their training data was collected. They also lack access to your private, proprietary data. RAG is an architectural pattern designed to solve both problems by grounding the model in up-to-date, relevant information at query time.

Instead of just sending a user's question to the LLM, a RAG system first retrieves relevant documents from a knowledge base (like a collection of PDFs, internal wiki pages, or support tickets) and then augments the user's prompt with this information before sending it to the LLM for generation.

This two-phase process can be visualized as follows:

Let's break down the two main phases shown in the diagram:

1. Ingestion (Offline): This is a preparatory step. Your source documents are loaded, broken down into smaller chunks, and converted into numerical representations called embeddings using an embedding model. These embeddings are then stored in a specialized Vector DB (Vector Database).
2. Retrieval & Generation (Online): When a user submits a query, it's also converted into an embedding. The system then performs a semantic search in the vector database to find the document chunks with embeddings that are most similar to the query's embedding. These retrieved chunks are combined with the original query into a new, augmented prompt, which is finally sent to the LLM.

To see this in action and understand the common failure points (even in non-adversarial settings), let's watch a short video.

The key insight here is that the quality of the LLM's output is now heavily dependent on the quality of the retrieved context. As the video notes, "bad retrieval is worse than no retrieval."

2. The Unchecked Trust: Introducing Indirect Prompt Injection

In a RAG architecture, a new trust boundary emerges. While we might scrutinize the user's input, the system implicitly trusts the content it retrieves from its own knowledge base. After all, it's "internal" data. This flawed assumption creates the "Back Door" for a new class of attacks.

Indirect Prompt Injection is an attack where malicious instructions are embedded within the documents in the knowledge base. When an unsuspecting user asks a relevant question, the system retrieves the poisoned document, and the malicious instructions are passed to the LLM as trusted context.

To understand this powerful attack vector, please watch the following video.

The video clearly illustrates the core threat: the LLM can't distinguish between the legitimate context it's supposed to summarize and the attacker's instructions hidden within that context. From the model's perspective, it's all just text.

3. Formalizing the Attack: Front Door vs. Back Door

To build a robust mental model for your demo, it's useful to formalize these attack vectors. The distinction between "Front Door" (direct) and "Back Door" (indirect) attacks is a powerful way to categorize these threats.

As the article explains, Back Door attacks are particularly dangerous due to their characteristics:

• Asynchronous: The attacker plants the payload now, and it gets triggered later by a legitimate user.
• Persistent: The payload remains in the knowledge base, ready to be activated multiple times.
• Scalable: A single poisoned document can affect every user whose query retrieves it.

This stands in stark contrast to the direct injections we studied previously, which are synchronous, transient, and affect only the current user's session.

4. The Delivery Mechanism: Corpus Poisoning

It's not enough to simply place a malicious file in a vast knowledge base. The attacker must ensure their payload is retrieved for specific, high-value queries. This is achieved through an attack on the retrieval process itself, known as Corpus Poisoning.

Since RAG relies on semantic similarity, attackers can't just use keywords. Instead, they must craft their malicious document so its embedding is mathematically close to the embedding of their target query.

The efficiency of this attack is alarming. You don't need to compromise the entire knowledge base.

From your background in infrastructure security, you might see a parallel here to a DNS cache poisoning attack. An attacker crafts a malicious record (the poisoned document and its embedding) that causes a common name resolution request (a user's query) to resolve to a malicious IP address (the attacker's payload), redirecting the victim without their knowledge.

Conclusion

In this lesson, we dissected the RAG architecture and uncovered its primary security flaw: the implicit trust placed in retrieved data. This "Back Door" enables persistent, scalable attacks that are much harder to detect and mitigate than the "Front Door" attacks we studied previously.

Key Takeaways:

• RAG Architecture: A two-phase process (Ingestion and Retrieval/Generation) that grounds LLMs in external data, solving the knowledge cutoff problem but introducing a new trust boundary.
• The Attack Surface: The knowledge base itself becomes a primary attack vector. Any data source that feeds your RAG system (document uploads, web scrapers, API integrations) is a potential entry point for malicious content.
• Indirect Prompt Injection: The core attack, where malicious instructions hidden in a retrieved document are executed by the LLM as if they were trusted context.
• Corpus Poisoning: The delivery mechanism. Attackers optimize a document's content to ensure its embedding is semantically close to a target query, guaranteeing its retrieval. This attack is highly efficient, requiring only a handful of poisoned documents.

We have now concluded our foundational module. You understand how LLMs process information, how direct prompt injection exploits in-context learning, and how RAG architectures create an even more potent vector for indirect injection.

In our next module, we will move from theory to practice. The first lesson, "Set up a local development environment with Ollama and a small language model (e.g., Llama 3 8B, Phi-3)," will guide you through installing the tools you need to build and test the vulnerable application for your demo.