Prompt Injection: Attacker Intent Classifications

Hello! Welcome to the third lesson in our module on prompt injection fundamentals.

In our last session, we drew a clear line between direct and indirect prompt injection, focusing on the delivery mechanism of the attack. You learned that direct attacks involve a malicious user targeting the application, while indirect attacks involve a third party poisoning a data source that a benign user's application later consumes.

Today, we'll shift our focus from the how to the why. We'll classify prompt injection attacks based on the attacker's ultimate goal or intent. To do this, we'll use a classic cybersecurity framework that will be very familiar from your work in SRE and infrastructure security: the CIA triad.

This lesson directly addresses the learning outcome: Classify prompt injection techniques by attacker intent: confidentiality, integrity, and availability. By the end, you will have a structured model for understanding the potential business impact of different prompt injection payloads.

The CIA Triad: A Framework for Impact

In information security, the CIA triad provides the foundational pillars for assessing threats:

• Confidentiality: Preventing the unauthorized disclosure of information.
• Integrity: Preventing unauthorized modification or destruction of information.
• Availability: Ensuring timely and reliable access to and use of information.

Prompt injection attacks are unique in that they can threaten all three pillars, often simultaneously. Understanding which pillar an attack targets helps in prioritizing defenses and communicating risk.

Let's begin with a video that masterfully maps real-world prompt injection exploits to each part of the CIA triad. The presenter, Johann Rehberger, is a prominent security researcher in this field, and his work provides a fantastic overview.

This video is based on a research paper which provides a more detailed, textual reference. It's a great resource to have on hand.

Now, let's break down each pillar with concrete examples.

1. Confidentiality Attacks: Stealing Secrets

Confidentiality attacks aim to extract sensitive information that the LLM has access to but should not reveal. This is often the most intuitive type of prompt injection attack.

Attacker's Goal:

• Leak the system prompt to understand the application's inner workings.
• Extract user data, conversation history, or documents from the context window.
• Steal API keys, database credentials, or other secrets embedded in the prompt.

The "Hacking LLM Apps & Agents" video shows several powerful techniques for exfiltrating data. One of the most common is through markdown injection.

This markdown rendering technique is a crucial one to understand. Since LLMs often output markdown, they can be tricked into creating <img> tags or hyperlinks that automatically send data to an external server when rendered by the user's browser or client application. For your demo, the "prompt leaking" payload you'll craft is a direct attack on confidentiality.

2. Integrity Attacks: Corrupting Behavior

Integrity attacks aim to manipulate the LLM's behavior to produce false, misleading, or harmful output, or to perform unauthorized actions.

Attacker's Goal:

• Hijack the LLM's original goal (e.g., turn a customer service bot into a scammer).
• Generate misinformation or propaganda.
• Bypass safety filters to produce offensive or dangerous content.
• Induce the LLM to call a tool or function with malicious parameters.

From your SRE perspective, this is analogous to an attacker gaining the ability to push unauthorized code or configuration changes that alter the system's logic.

Let's watch another segment from the same video that shows clear examples of integrity violations.

Another way to frame this is through the lens of "Behavioral Manipulation." A concise guide offers this taxonomy.

The "goal hijacking" payload you will develop for your demo—where you make the chatbot give a fake financial recommendation—is a quintessential integrity attack.

3. Availability Attacks: Disrupting Service

Availability attacks are designed to make the LLM application unresponsive, unstable, or unusable. As an SRE, this is your home turf: preventing Denial of Service (DoS) and ensuring system reliability. With LLMs, the attack vectors are computational rather than network-based.

Attacker's Goal:

• Trick the LLM into a recursive loop or a long-running computation, consuming expensive GPU resources and blocking other users.
• Cause the model to refuse to process legitimate requests.
• Persistently modify the LLM's memory or state to cause future failures.

The "Trust No AI" paper has a concise section on this topic.

A fascinating example of a persistent availability attack involves manipulating the LLM's memory.

This type of attack is particularly relevant given your infrastructure background. The cost of LLM inference is high. An attacker who can induce expensive, long-running, or recursive computations is launching a resource-exhaustion DoS attack that directly translates to financial cost.

Conclusion

In this lesson, we've used the CIA triad as a powerful lens to classify prompt injection attacks based on their intended impact. This framework moves beyond the technical implementation and helps us reason about the business risk of these vulnerabilities.

Key Takeaways:

• The CIA triad (Confidentiality, Integrity, Availability) provides a structured way to categorize prompt injection attacks by their goal.
• Confidentiality attacks aim to steal information. Key techniques include prompt leaking and data exfiltration via methods like markdown rendering.
• Integrity attacks aim to alter behavior or corrupt output. Key techniques include goal hijacking, generating misinformation, and bypassing safety filters.
• Availability attacks aim to disrupt service. Key techniques include inducing resource exhaustion (e.g., infinite loops) or causing refusal of service.
• A single payload can often impact multiple pillars. For instance, an attack that hijacks an agent to call an external API (Integrity) could also exfiltrate data (Confidentiality) and enter a recursive loop (Availability).

Next Up:

We now have two orthogonal ways to classify prompt injection: by delivery method (direct vs. indirect) and by intent (CIA). With this solid theoretical foundation, we are ready to turn our attention to the application you will build. In the next lesson, we will analyze our vulnerable target application against the OWASP LLM01 criteria to identify its specific weaknesses, setting the stage for crafting our 10 attack payloads.