AI Security Prompt Injection Demo

Analyze how tokenization and next-token prediction create inherent vulnerabilities in LLMs.

Describe the roles of system prompts, user prompts, and the context window in processing LLM requests.

Explain how in-context learning and few-shot prompting are the mechanisms that enable prompt injection attacks.

Diagram a Retrieval-Augmented Generation (RAG) pipeline and identify the new attack surfaces it introduces, particularly for indirect injection.

Set up a local development environment with Ollama and a small language model (e.g., Llama 3 8B, Phi-3).

Explain the core components of LangChain: chains, prompt templates, and output parsers.

Build a minimal LangChain RAG chatbot wrapped in a FastAPI endpoint.

Identify and document the deliberate vulnerabilities in the target app: unvetted RAG data, no input validation, no output filtering, and a permissive system prompt.

Test the target application end-to-end to confirm it is functional and exploitable.

Summarize the OWASP Top 10 for LLM Applications, focusing on the LLM01: Prompt Injection category.

Distinguish between direct and indirect prompt injection with concrete examples.

Classify prompt injection techniques by attacker intent: confidentiality, integrity, and availability.

Map each of the 10 planned attack payloads to its relevant prompt injection technique and intent.

Execute a basic instruction override attack to make the LLM ignore its system prompt.

Execute a jailbreak attack using role-playing (e.g., DAN-style) to bypass safety guidelines.

Execute a prompt leaking attack to extract the system prompt and other sensitive context from the application.

Execute a few-shot injection attack that manipulates the model's behavior by providing fabricated examples in the prompt.

Execute a context manipulation attack using delimiter injection to alter the model's interpretation of the prompt structure.

Execute a goal hijacking attack that redirects the chatbot to produce a fabricated, business-critical output (e.g., fake financial recommendation).

Execute a payload splitting attack that breaks malicious instructions across multiple inputs to evade simple filters.

Execute an obfuscation attack using Base64 or character substitution to bypass naive pattern-matching filters.

Execute an indirect prompt injection attack by poisoning the context data used by the RAG pipeline.

Execute a multi-turn social engineering attack that gradually co-opts the model's behavior across a conversation.

Explain the role of automated scanners like Garak in an LLM security testing workflow.

Install and configure Garak to target the local Ollama-backed chatbot.

Run a scan using built-in probes relevant to prompt injection and interpret the results.

Implement a basic input sanitizer that detects and blocks known injection patterns (e.g., using regex).

Explain what Guardrails AI is and how it adds programmable, runtime validation to LLM interactions.

Define output guards that enforce response format and content constraints (e.g., preventing prompt leaks).

Analyze the trade-offs of Guardrails AI, considering latency overhead, configuration complexity, and coverage limits.

Explain why output filtering is a critical final layer in a defense-in-depth strategy.

Implement a multi-strategy output filter to redact sensitive information and detect indicators of a successful attack.

Evaluate the filter's effectiveness against all 10 attack payloads, documenting which malicious outputs are blocked.

Analyze the edge cases where output filtering fails, such as steganographic encoding or heavily paraphrased leaks.

Articulate a defense-in-depth strategy that combines input, runtime, and output controls for comprehensive protection.

Explain the role of observability in LLM security for threat detection and incident response.

Create Langfuse dashboards to visualize attack patterns and the trigger rates of defense mechanisms.

Use Langfuse traces to perform a post-incident analysis on a simulated prompt injection attack, identifying the root cause and impact.

Integrate all defense and monitoring layers into the final FastAPI application.

Synthesize a before-and-after comparison report showing attack success rates with and without the defense layers.

Prepare a concise architecture diagram illustrating the complete data flow, including all attack, defense, and monitoring components.

Storyboard the demo's narrative arc, scripting talking points for each phase: problem, threat landscape, attacks, defenses, and monitoring insights.

Draft concise answers to common business-focused questions about AI security, translating technical trade-offs into business impact.