In our last lesson, we took the first step in a hacker's reconnaissance journey: translating a human-readable domain name into a machine-readable IP address using DNS. Now that your browser knows where to send its request, it needs to establish a reliable communication channel to the server at that address. This lesson moves one step further down the networking stack to explore how that connection is made.
Today, we will focus on the Transmission Control Protocol (TCP) and its famous three-way handshake. You will learn how this fundamental process works and, more importantly for your bug bounty goals, how understanding it allows you to perform network reconnaissance. By the end of this lesson, you will be able to describe the handshake mechanism and explain how it's leveraged to discover open ports and services on a target system.
Setting the Stage: Connection-Oriented vs. Connectionless
Before diving into the handshake itself, it's essential to understand why it's needed. On the internet, data is primarily sent using one of two transport protocols: TCP or UDP. They serve different purposes.
- UDP (User Datagram Protocol): This is a "fire and forget" protocol. It's fast and efficient but doesn't guarantee delivery or order. It's like sending a postcard—you drop it in the mail and hope it gets there, but there's no confirmation. UDP is great for services like DNS lookups or online gaming, where speed is more critical than perfect reliability.
- TCP (Transmission Control Protocol): This protocol is all about reliability. It ensures that all data packets arrive, are in the correct order, and are error-free. To achieve this, TCP establishes a formal connection before any data is sent. It's like making a phone call—you wait for the other person to pick up and say "hello" before you start the conversation. Web browsing (HTTP/HTTPS), file transfers (FTP), and email all rely on TCP.
Since web applications, our primary targets in bug bounty hunting, depend on TCP, understanding how it establishes a connection is fundamental.
Networking for Ethical Hackers - TCP, UDP, and the Three-Way Handshake (Re-Up)
To solidify this concept, watch this introductory segment from The Cyber Mentor's video, "Networking for Ethical Hackers."
Pay close attention to the explanation of TCP vs. UDP. The video highlights which types of applications use each protocol, which will help you understand why the three-way handshake is so central to web communication.
The Three-Way Handshake: A Digital Conversation
The process TCP uses to establish a connection is called the three-way handshake. It's a sequence of three messages exchanged between the client (e.g., your browser) and the server. This "conversation" ensures both sides are ready to communicate.
To understand this, we first need to know about TCP flags, which are one-bit fields in the TCP header that signal the state of a connection. For the handshake, two are critical:
- SYN (Synchronize): Used to initiate a connection. It essentially says, "I want to start a connection with you."
- ACK (Acknowledge): Used to confirm receipt of a packet. It says, "I received your message."
The handshake unfolds in three steps:
-
SYN: The client sends a TCP packet to the server with the SYN flag set. This packet includes a random sequence number (let's call it
X) that will be used to track the data packets.- Client -> Server: "I'd like to connect. My starting sequence number is X."
-
SYN/ACK: If the server is listening and accepts the connection, it responds with a packet where both the SYN and ACK flags are set. It acknowledges the client's sequence number (by sending
X+1) and sends its own random sequence number (let's call itY).- Server -> Client: "I acknowledge your request (X+1). I'm ready to connect too. My starting sequence number is Y."
-
ACK: The client receives the server's response and completes the connection by sending a final packet with the ACK flag set. It acknowledges the server's sequence number (by sending
Y+1).- Client -> Server: "I acknowledge your readiness (Y+1). The connection is established. Let's start sending data."
Once the third step is complete, the TCP connection is open, and the client and server can begin exchanging application data (like an HTTP request).
Networking for Ethical Hackers - TCP, UDP, and the Three-Way Handshake (Re-Up)
The Cyber Mentor provides a clear, concise walkthrough of this process, followed by a fantastic real-world demonstration using the packet analysis tool Wireshark.
First, watch the theoretical explanation of the three steps. Next, watch the practical demonstration where he captures the handshake in Wireshark. Notice how the packets in the capture are labeled with [SYN], [SYN, ACK], and [ACK], exactly matching the three steps we just discussed. This visual confirmation is invaluable for making the concept concrete.
The Handshake's Role in Reconnaissance: Port Scanning
So, why is this essential for a bug bounty hunter? Because this predictable behavior can be used to determine which services are running on a target machine. This process is called port scanning. Every service on a server (like a web server on port 443 or a database on port 3306) listens for incoming connections. By attempting a handshake, we can see if anyone "picks up the phone."
There are two primary ways we can leverage the handshake for this purpose:
1. TCP Connect Scan (Full Handshake)
This is the most straightforward type of scan. The scanner (your tool) attempts to complete the full three-way handshake with a specific port on the target.
- If the handshake completes successfully (SYN, SYN/ACK, ACK), the port is open.
- If the server responds with a packet that has the RST (Reset) flag set, it means the connection is refused, and the port is closed.
- If there is no response, the port is likely filtered (e.g., blocked by a firewall).
This method is reliable but "noisy." Since a full connection is established, it's easily logged by the target system, making your reconnaissance activities obvious.
2. TCP SYN Scan (Half-Open Scan)
This is a more popular and stealthier technique. Instead of completing the handshake, the scanner stops halfway.
- The scanner sends a SYN packet to the target port.
- If the server responds with a SYN/ACK, the port is open.
- Instead of sending the final ACK to complete the connection, the scanner sends an RST packet to tear it down immediately.
Because the full connection is never established, this type of scan is less likely to be logged by the application running on the server. This makes it a preferred method for initial reconnaissance, as it helps you map out the attack surface without immediately alerting the target.
Network Scanning - TCP Flags & TCP 3-Way Handshake
HackerSploit's video on network scanning provides more detail on the flags and the scanning techniques themselves.
First, watch the section covering the different TCP flags. Understanding RST (Reset) and FIN (Finish) in addition to SYN and ACK is crucial for understanding different scanning behaviors. Next, watch the explanation of how the handshake is used in network scanning. The video explicitly contrasts a full scan (completing the handshake) with a stealth scan (using SYN and then RST).
By using tools like nmap (which we will cover later), a bug bounty hunter can perform these scans across thousands of ports on a target IP address in seconds. The results provide a map of potential entry points. An open port 80 or 443 points to a web server, port 22 suggests an SSH server, and port 21 indicates an FTP server. Each open port is a door that you can investigate for vulnerabilities.
Conclusion
In this lesson, we've moved from finding a target's address to knocking on its doors. You now understand that for reliable web communication, TCP establishes a connection using a three-step process: SYN, SYN/ACK, and ACK. This isn't just a technical formality; it's a mechanism that can be actively probed for information.
Key Takeaways:
- TCP is a connection-oriented protocol that guarantees reliable data delivery, making it the foundation for web traffic.
- The three-way handshake (SYN, SYN/ACK, ACK) is the process TCP uses to establish a connection before any data is sent.
- Understanding this handshake is crucial for network reconnaissance, as it forms the basis of port scanning.
- A TCP Connect Scan completes the full handshake and is reliable but easily detected.
- A TCP SYN Scan (or "stealth scan") only completes half the handshake, allowing you to discover open ports more discreetly.
You now have a conceptual understanding of how to discover active services on a target. In our next lesson, we will finally look at what happens after this connection is made: we will deconstruct the HTTP requests and responses that are exchanged over the established TCP channel.