TCP 3-Way Handshake: Establishing Reliable Connections

1. The Core Problem: Building Reliability on an Unreliable Foundation

As we've discussed, the Internet Protocol (IP) is inherently unreliable. It makes a "best effort" to deliver packets, but it offers no guarantees. Packets can be lost, duplicated, or arrive out of order.

TCP's primary job is to create a reliable, connection-oriented service on top of this unreliable IP layer. To do this, it needs a mechanism to track every single byte of data sent and confirm its receipt. The foundation of this mechanism is built on two key fields in the TCP header:

• Sequence Number (SEQ): A 32-bit number that tracks the bytes sent. If a TCP segment contains 100 bytes of data, the sequence number for the next segment will be incremented by 100. It doesn't count packets; it counts bytes.
• Acknowledgment Number (ACK): A 32-bit number used by the receiver to inform the sender which byte it expects to receive next. If the receiver has successfully received all bytes up to byte 999, it will send an ACK number of 1000.

Before any application data can be exchanged, both sides must agree on a starting point for their sequence numbers. This process of agreement and synchronization is the primary purpose of the three-way handshake.

To kick things off, let's watch a video that elegantly explains how these sequence and acknowledgment numbers work to track bytes.

2. The Three-Way Handshake: A Step-by-Step Breakdown

The handshake process involves the exchange of three specific TCP segments to synchronize the sequence numbers and verify that both ends are ready. Each of these segments has special one-bit flags set in the TCP header to define its purpose. The two most important flags for the handshake are:

• SYN (Synchronize): Indicates a request to initiate and synchronize a connection.
• ACK (Acknowledge): Indicates that the Acknowledgment Number field is significant.

Here are the three steps:

Step 1: Client sends SYN

The client (the "active opener") initiates the connection by sending a TCP segment to the server.

• Flags: The SYN flag is set to 1.
• Sequence Number: This field contains the client's Initial Sequence Number (ISN), a randomly chosen starting value. Let's call it ISN_C.
• What it means: "Hello server, I'd like to establish a connection. My starting sequence number is ISN_C."
• State Change: The client transitions from the CLOSED state to the SYN_SENT state.

Step 2: Server responds with SYN-ACK

When the server's OS receives the SYN packet on a listening port, it responds if it's able to accept a new connection.

• Flags: Both the SYN and ACK flags are set to 1.
• Sequence Number: This field contains the server's own randomly chosen Initial Sequence Number, ISN_S.
• Acknowledgment Number: This field is set to ISN_C + 1. This is critical. By incrementing the client's sequence number, the server is explicitly acknowledging receipt of the client's SYN packet.
• What it means: "I hear you, client. I'm also ready to connect. My starting sequence number is ISN_S, and I am acknowledging your request by confirming I expect your next byte to be ISN_C + 1."
• State Change: The server transitions from LISTEN to SYN_RCVD (SYN Received).

Step 3: Client sends ACK

The client receives the server's SYN-ACK. The connection is now confirmed from the client's perspective. The client sends one final segment to complete the handshake.

• Flags: The ACK flag is set to 1.
• Sequence Number: This is now ISN_C + 1, as acknowledged by the server.
• Acknowledgment Number: This is now ISN_S + 1, acknowledging the server's SYN packet.
• What it means: "Got it, server. I acknowledge your sequence number. The connection is established."
• State Change: The client transitions to ESTABLISHED. When the server receives this final ACK, it also transitions to ESTABLISHED.

At this point, the handshake is complete. A reliable, bi-directional communication channel is open, and both sides can begin sending application data.

For a clear, step-by-step reading on this process, the following article is very helpful.

3. A Deep Dive with Wireshark

Theory is great, but let's go to the "root level" you're interested in and see what these packets actually look like on the wire. The best way to do this is with a packet analyzer like Wireshark.

The following video provides an excellent, practical walkthrough of a real TCP handshake captured in Wireshark. It will solidify your understanding of how sequence numbers, acknowledgment numbers, and flags appear in a live capture.

One interesting detail from the video is the concept of the "phantom byte" or "ghost byte". The SYN and FIN (finish) flags each consume one number in the sequence space, even though they don't carry any actual application data. This is why the acknowledgment for a SYN with sequence number X is X+1. This ensures that the SYN packet itself can be uniquely acknowledged.

4. Under the Hood: Server State and Kernel Queues

Let's connect this back to what's happening on the server. In the previous lesson, we learned that a server application calls listen() and then accept(). The three-way handshake fits neatly between these two calls.

When the server is in the LISTEN state, its kernel manages two important queues:

1. The SYN Queue (or Half-Open Connection Queue): When the kernel receives the initial SYN packet (Step 1), it creates a small record for this pending connection and places it in the SYN queue. The connection is considered "half-open." The kernel then sends the SYN-ACK (Step 2).

2. The Accept Queue (or Full Connection Queue): When the kernel receives the final ACK from the client (Step 3), the connection is now fully established. The kernel moves the connection's record from the SYN queue to the Accept queue.

The server application's accept() call simply picks up a fully formed connection from the front of the Accept queue. If the Accept queue is empty, the accept() call will block, waiting for a new connection to become fully established.

This separation is also the basis for a common type of Denial-of-Service attack called a SYN flood, where an attacker sends a high volume of SYN packets but never completes the handshake, overwhelming the server's SYN queue.

Conclusion

You've now dissected the intricate dance of the TCP three-way handshake. It's not just a simple "hello"; it's a carefully choreographed process designed to build a reliable communication channel on an unreliable network.

Key Takeaways:

• The primary purpose of the three-way handshake is to synchronize sequence numbers (ISNs) between the client and server.
• The process consists of three steps: SYN, SYN-ACK, and ACK.
• The SYN flag initiates a connection, and the ACK flag acknowledges received data (or a SYN).
• Sequence numbers track the bytes of data sent, while acknowledgment numbers confirm their receipt. This is the foundation of TCP's reliability.
• On the server, the kernel uses a SYN queue for half-open connections and an Accept queue for fully established connections that are ready for the application to accept().

Preview of the Next Lesson:

TCP provides incredible reliability, but that reliability comes with overhead: the three-way handshake, sequence number tracking, and acknowledgments. What if you don't need all that? What if speed is more important than perfect reliability? In our next lesson, we will compare TCP with its sibling protocol, UDP (User Datagram Protocol), and explore the trade-offs and use cases for each.