Building the Request Interception and Parsing Component

Hello! Welcome back.

In our last lesson, we evaluated the trade-offs between the reverse proxy, transparent bridge, and cloud-hosted WAF deployment modes. We concluded that the reverse proxy model, while complex, offers the highest degree of control and visibility, making it an excellent archetype for our deep dive into security product architecture.

Today, we begin designing the core of that reverse proxy WAF. We'll focus on the very first component that touches incoming traffic, addressing the learning outcome: Design the request interception and parsing component of a reverse-proxy WAF, including HTTP connection handling and request normalization.

This is a critical first step. An attacker's primary goal is to evade detection, and the easiest way to do that is to trick the WAF into misinterpreting the request. If the WAF and the backend server "see" two different versions of the same request, a vulnerability is born. Our goal today is to design a component that prevents this by reliably intercepting, parsing, and sanitizing every request before inspection.

1. Connection Handling: From TCP to HTTP

A WAF operating as a reverse proxy is a Layer 7 device. This means it must fully terminate the client's connection and establish a new one to the backend server. Let's break down what this entails.

First, watch this short segment to refresh the distinction between Layer 4 and Layer 7 proxies.

This ability to decrypt and see the plain-text HTTP traffic is the entire reason a reverse proxy WAF is effective.

From an architectural standpoint, this introduces several design considerations:

• TCP Connection Management: A production-grade WAF must handle tens or hundreds of thousands of concurrent TCP connections. This is the classic C10k problem. Modern, high-performance proxies (and by extension, WAFs) use event-driven, non-blocking I/O APIs like epoll (Linux) or kqueue (BSD) to manage this efficiently without dedicating a thread to each connection.
• TLS Termination Performance: The TLS handshake, particularly the asymmetric cryptography part, is CPU-intensive. The WAF's TLS termination layer must be highly optimized. This often involves leveraging hardware acceleration features in modern CPUs (like Intel's AES-NI instruction set) or using highly optimized TLS libraries.
• Certificate and Key Management: To terminate a TLS session for yourapp.com, the WAF must possess the TLS certificate and, critically, the private key for yourapp.com. Securely storing, distributing, and rotating these keys across a fleet of WAF instances is a significant system design challenge in itself.

2. Request Parsing: Building a Structured Representation

Once the WAF has the decrypted, raw HTTP/1.1 byte stream, it must parse it into a structured format that security modules can understand. This is harder and more security-critical than it sounds. HTTP/1.1 is a text protocol with ambiguous framing rules, making it a fertile ground for "request smuggling" attacks where the proxy and the backend disagree on where one request ends and the next begins.

The robust architectural solution is to parse the request once into an unambiguous internal representation. All subsequent processing happens on this clean, structured data, not the raw, potentially malicious byte stream.

Let's study a real-world, high-performance implementation of this principle: HAProxy's HTX layer.

The HTX article gives us the blueprint for our parsing component:

1. Ingest Raw Data: Read the raw HTTP request from the TCP socket into a buffer.
2. Parse into Blocks: Create a parser that walks through the buffer and transforms the byte stream into a structured, linked list of "blocks."
   • Start-Line Block: Contains method, URI, and HTTP version.
   • Header Blocks: One block for each {name, value} pair.
   • End-of-Headers Marker: An explicit block that separates headers from the body.
   • Data Blocks: One or more blocks containing the request body, completely stripped of any transport-level encoding (like HTTP chunking).
3. Provide a Safe API: The rest of the WAF (the inspection engine) interacts only with this structured representation through a safe API that prevents modification bugs from re-introducing protocol-level vulnerabilities. For example, an API to change a header value would ensure the new value contains no CRLF characters, making header injection impossible.

Parsing the Request Body

The request body itself can have various Content-Type formats that require their own specialized parsers. Our WAF's parsing component would need to be modular to handle these.

• application/x-www-form-urlencoded: Key-value pairs, relatively simple to parse.
• multipart/form-data: Complex format with boundaries, used for file uploads. The parser must carefully handle the boundary definitions to avoid being tricked.
• application/json: Requires a JSON parser, which is often implemented as a state machine.

This leads to a crucial design choice, as shown in tools like ModSecurity: you must be able to enable or disable body processing. Parsing a large request body (e.g., a file upload) is resource-intensive, so a WAF administrator needs the control to enable it only when necessary.

3. Request Normalization: Defeating Evasion

Parsing gives us structure, but the content of that structure is still untrusted. Attackers will use various encoding and obfuscation methods to hide attack payloads. Normalization is the process of repeatedly transforming input data into its simplest, canonical form before it's inspected.

Let's explore the core concepts of normalization and why it's so critical for preventing bypasses.

As the text shows, a normalization engine is essentially a pipeline of transformation functions. Common functions include:

• URL decoding (%20 -> space)
• HTML entity decoding (< -> <)
• Base64 decoding
• Whitespace collapsing
• Lowercase conversion

The goal is to ensure that a payload like <ScrIpt%20> is reduced to the canonical form <script > before it hits the rule engine.

The Impedance Mismatch Vulnerability

This is the single most important concept in WAF bypass. A vulnerability is created when the WAF normalizes a request differently from the backend application server. An attacker can craft a payload that looks benign to the WAF but is interpreted as malicious by the backend.

HTTP Parameter Pollution (HPP) is the classic example. A request like ?id=value1&id=value2 is ambiguous. How does the server interpret it?

• PHP/Apache might use the last occurrence: id is value2.
• ASP.NET/IIS might concatenate them: id is value1,value2.
• Java/Tomcat might use the first: id is value1.

Now, imagine a WAF configured to inspect only the first parameter. If the backend is a PHP server that uses the last parameter, an attacker can bypass the WAF with a request like: ?id=SAFE_VALUE&id=MALICIOUS_PAYLOAD.

This video explains the concept beautifully.

This leads to our final design principle for the parsing and normalization component: The WAF's parsing and normalization logic must be configurable to precisely match the behavior of the backend it is protecting. This is not a "one size fits all" problem. A WAF protecting a PHP application may need different HPP settings than one protecting an ASP.NET application.

Conclusion

We've now designed the complete front door of our reverse proxy WAF. We've established how it intercepts traffic and how it meticulously dismantles and sanitizes requests to create a clean, trustworthy representation for security analysis.

Key Takeaways:

• Connection Handling: A reverse proxy WAF acts as a full TLS-terminating proxy, which requires efficient connection management and secure handling of cryptographic keys.
• Request Parsing: The most robust design pattern is to parse the raw, ambiguous HTTP stream once into a clean, structured internal representation (like HAProxy's HTX). This prevents request smuggling and other parsing-level attacks.
• Request Normalization: This is a critical security stage that applies a chain of transformations (decoding, case changes, etc.) to defeat payload obfuscation. The order of these transformations is crucial.
• Impedance Mismatch: The root of many WAF bypasses lies in differences between how the WAF and the backend server interpret the same request. The WAF's parsing and normalization logic must be aligned with the backend's behavior.

Preview of Next Lesson:

Now that we have a clean, normalized, and structured representation of the HTTP request, what do we do with it? In our next lesson, we will address the outcome: Design a modular inspection pipeline that chains multiple security modules (e.g., signature, anomaly) with configurable execution order. We will design the engine that actually looks for the attacks.