Packet Capture and Analysis with tcpdump

1. What is tcpdump?

tcpdump is a command-line packet analyzer. It allows you to intercept and display TCP/IP and other packets being transmitted or received over a network. Since it runs in the terminal, it's an indispensable tool for troubleshooting on servers, in containers, or any environment where you don't have a graphical user interface.

While tools like Wireshark provide a rich graphical interface for packet analysis, the standard professional workflow often involves using tcpdump on a remote server to capture the traffic and then transferring the capture file to your local machine for in-depth analysis with Wireshark.

Let's begin with a foundational video that introduces the tool and its basic operation.

As you saw, one of the first things to do is identify which network interface you want to listen on.

# List available interfaces
sudo tcpdump -D




# A basic capture on the 'eth0' interface (press Ctrl+C to stop)
sudo tcpdump -i eth0

The output can be a bit overwhelming and scroll by quickly. This is where options become critical. The How to Handle Network Troubleshooting with tcpdump article provides a great reference table of these options.

2. Interpreting Packet Information

When you run tcpdump, each line of output represents a single packet. Understanding its structure is key to analysis.

A standard line of TCP packet output looks something like this:

10:15:32.123456 IP 192.168.1.100.45678 > 93.184.216.34.80: Flags [S], seq 12345, win 65535, options [mss 1460], length 0

Let's break that down:

• 10:15:32.123456: Timestamp
• IP: Protocol (Layer 3)
• 192.168.1.100.45678: Source IP address and source port
• 93.184.216.34.80: Destination IP address and destination port
• Flags [S]: TCP Flags. [S] is SYN, [.] is ACK, [P] is PUSH, [F] is FIN, [R] is RST.
• seq 12345: The TCP sequence number of the data in this packet.
• win 65535: The TCP window size (how much data the sender is willing to receive).
• length 0: The length of the payload data in bytes.

The video you watched earlier provided a great explanation of these fields. For a deeper dive, especially on how tcpdump displays TCP handshakes, sequence numbers, and even application-layer data like DNS, the next segment is very helpful.

3. The Power of Filtering

Capturing all traffic is rarely useful. The real power of tcpdump comes from its filtering capabilities, which use the Berkeley Packet Filter (BPF) syntax. This lets you zero in on exactly the traffic you're interested in.

Filters are composed of expressions and can be combined with logical operators like and, or, and not.

Here are the most common filter types:

Combining Filters:

To troubleshoot a web server at 10.10.5.20, you might want to see all HTTP/HTTPS traffic to it, but exclude health checks from your load balancer at 10.10.1.1:

sudo tcpdump -i eth0 -nn 'host 10.10.5.20 and (port 80 or port 443) and not host 10.10.1.1'

Notice the single quotes around the filter expression. This is important to prevent the shell from misinterpreting special characters like parentheses.

The following video provides excellent hands-on examples of building these filters.

4. The Professional Workflow: Saving and Reading Captures

Watching live traffic is useful for quick checks, but for serious analysis, you'll want to save the capture to a file. This creates a .pcap (packet capture) file that you can analyze later with tcpdump or a graphical tool like Wireshark.

• -w <filename>: Write packets to a file.
• -r <filename>: Read packets from a file.

Scenario: You suspect an issue with a containerized application on a remote server.

1. On the server, start a capture and save it to a file. Use filters to keep the capture size manageable.

   
   
   
   # Capture up to 5000 packets destined for port 3306 (MySQL) and save to a file
   sudo tcpdump -i eth0 -c 5000 -w mysql_traffic.pcap 'dst port 3306'
   

2. Transfer the file to your local machine using scp or another method.

   scp user@remote-server:~/mysql_traffic.pcap .
   

3. On your local machine, you can now analyze it.

   
   
   
   # Read the file with tcpdump
   tcpdump -r mysql_traffic.pcap -nn
   
   
   
   
   # Or, for much deeper analysis, open it with Wireshark
   wireshark mysql_traffic.pcap
   

This workflow is demonstrated very effectively in the next video.

5. Practical Troubleshooting Scenario

Let's apply what we've learned.

Problem: A newly deployed microservice (orders-service) is failing. The logs show "Connection Timed Out" when trying to reach the postgres-db service at IP 10.244.1.50 on port 5432.

Your Goal: Determine where the communication is breaking down.

You ssh into the node where the orders-service pod is running.

Step 1: Formulate a filter.
You need to see traffic between your service and the database. The source IP will be the pod's IP (let's say 10.244.2.100) and the destination is the database.

# Filter for traffic between the service and the DB on the correct port
sudo tcpdump -i any -nn 'host 10.244.2.100 and host 10.244.1.50 and port 5432'

(Using -i any is often useful in complex environments like Kubernetes to capture on all interfaces, though it can be noisy.)

Step 2: Analyze the output.

• Scenario A: You see nothing. If you restart the orders-service and still see no packets matching your filter, the application might be misconfigured (e.g., wrong DB address) and isn't even attempting a connection. The problem is likely not the network itself.

• Scenario B: You see SYN packets, but no replies.

  14:01:10.123 IP 10.244.2.100.54321 > 10.244.1.50.5432: Flags [S], ...
  14:01:11.123 IP 10.244.2.100.54321 > 10.244.1.50.5432: Flags [S], ... (retransmit)
  14:01:13.123 IP 10.244.2.100.54321 > 10.244.1.50.5432: Flags [S], ... (retransmit)
  

  This indicates your service is trying to start a TCP connection (SYN), but it's getting no response. This strongly suggests a firewall issue (like a Network Policy in Kubernetes or a security group in AWS) or a routing problem preventing the packets from reaching the database server.

• Scenario C: You see SYN packets, then RST packets in reply.

  14:02:20.456 IP 10.244.2.100.54322 > 10.244.1.50.5432: Flags [S], ...
  14:02:20.457 IP 10.244.1.50.5432 > 10.244.2.100.54322: Flags [R.], ...
  

  The [R] flag stands for Reset. This means the packet reached the destination server, but the server's OS actively rejected the connection on that port. This usually means no process is listening on port 5432 on the DB server. The database process is likely crashed or not running.

Conclusion

You've now learned how to use tcpdump to look "under the hood" of your network. This skill transforms network troubleshooting from guesswork into a data-driven forensic process.

Key Takeaways:

• tcpdump is a command-line tool for capturing network packets, essential for server and container environments.
• Always start with a filter. The BPF syntax allows you to isolate traffic by host, port, net, and protocol.
• The professional workflow is to capture with tcpdump -w on the server and analyze with Wireshark locally.
• By analyzing TCP flags like [S] (SYN), [R] (Reset), and the absence of replies, you can diagnose a wide range of connectivity issues.

Preview of the Next Lesson:
We've seen the packets, the protocols, and the handshakes in action. In our next lesson, we will begin the "Networking for DevOps Engineers" module by formalizing this knowledge. The learning outcome will be to explain the OSI and TCP/IP models and map common protocols (HTTP, DNS, TCP) to their layers. This will provide the theoretical framework to understand why the network behaves the way it does.