Understanding Key HTTP Headers

The Anatomy of a Message: What are Headers?

If the HTTP method is the verb and the URL path is the noun, then headers are the adjectives and adverbs that provide essential context. They are key-value pairs that travel with every HTTP request and response, carrying information about the client, the server, the message body, and the connection itself.

Headers provide the instructions and context needed for the client and server to have a coherent conversation. A clear understanding of these headers is essential before we begin intercepting and modifying them with Burp Suite.

For a great verbal overview of the most common headers, the HackerSploit channel offers a clear walkthrough.

Now, let's explore the specific headers for this lesson in more detail, focusing on their security implications. The TryHackMe guide, "HTTP in detail," provides concise definitions that we can build upon.

Key Headers for the Ethical Hacker

With those definitions as our foundation, let's look at each header from a security analyst's perspective. Your goal in CTFs and bug bounties is often to find where the application makes flawed assumptions based on the data in these headers.

1. The Host Header

• Purpose: The Host header specifies the domain name of the server the client is trying to reach. This is essential for modern web servers, which often host multiple websites on a single IP address (a practice known as virtual hosting). The server uses the Host header to know which website's content to serve.
• Security Relevance: If an application blindly trusts the Host header, it can be vulnerable to Host Header Injection. For example, a password reset link might be generated using the value from the Host header. By manipulating it to point to an attacker-controlled domain, you could trick the application into sending a user's password reset link to you instead.

2. The User-Agent Header

• Purpose: This header is a string that identifies the client's browser, operating system, and version. It allows the server to deliver content tailored to the specific client (e.g., a mobile version of a site for a mobile browser).
• Security Relevance: Changing the User-Agent string is a common technique. You might switch to a mobile user agent to see if the mobile site has weaker security controls. Sometimes, applications grant special access or reveal debugging information to specific user agents, like search engine crawlers. A CTF challenge might hide a flag that is only visible if you claim to be a specific, non-standard user agent.

This CTF write-up provides a perfect example of spoofing the User-Agent to solve a challenge.

3. The Content-Type Header

• Purpose: When sending data in the body of a request (like with a POST method), the Content-Type header tells the server how to interpret that data. Common values include application/x-www-form-urlencoded for standard web forms and application/json for API requests.
• Security Relevance: Mismatched or manipulated Content-Type headers can be used to bypass security controls. For instance, a Web Application Firewall (WAF) might be configured to inspect JSON data but not XML. By sending a malicious payload with a Content-Type: application/xml header, you might evade detection. As you'll see in the next example, sometimes simply including the correct header is necessary for the application to process your request at all.

Let's look at another example from the same CTF write-up.

4. The Cookie Header

• Purpose: Since HTTP is stateless, servers use cookies to remember you between requests. The server first sends a Set-Cookie header in a response. Your browser stores this cookie and then sends it back to the server in a Cookie header on all subsequent requests to that domain. This is how websites keep you logged in.
• Security Relevance: Cookies are a prime target. The Cookie header is where you'll find session tokens. If you can steal another user's cookie, you can often impersonate them. Furthermore, sometimes cookies contain other data, like user roles (e.g., isAdmin=false). If the application trusts this data without server-side verification, you might be able to change it to isAdmin=true and escalate your privileges.

Our final CTF example shows how modifying a cookie's value can lead directly to a flag.

To see these headers demonstrated in common tools, the Traversy Media HTTP Crash Course provides an excellent practical walkthrough.

Conclusion

In this lesson, we moved beyond the HTTP method to explore the rich metadata contained in HTTP headers. They are fundamental to how the web functions and are a primary area of interest for any security tester.

Here are the key takeaways:

• Headers provide context: They are key-value pairs that tell the client and server about each other and about the message being sent.
• Host: Tells the server which website you want, enabling multiple sites on one IP. Can be a target for injection attacks.
• User-Agent: Identifies your browser/OS. Spoofing it can bypass client-side controls or reveal hidden content.
• Content-Type: Describes the data in the request/response body. Manipulating it can be used to bypass filters, and sending the correct type is often required for an application to function.
• Cookie: Carries state information, most importantly session identifiers. Manipulating cookies is a classic technique for session hijacking and privilege escalation.

So far, we've discussed the structure of HTTP messages, their methods, and their headers. In the next lesson, we will finally bring Burp Suite into the picture by explaining the concept of a web proxy. You will learn how a proxy sits between your browser and the web server, allowing you to intercept, inspect, and—most importantly—modify these very headers before they reach their destination.