Port Scanning Techniques: A Comparative Analysis

Hello! Welcome back to our course on architecting security products.

In the previous lesson, we designed the high-level architecture for a scan orchestration engine. We established a robust control plane for managing targets, scheduling scans, and coordinating a fleet of distributed workers using a lease-based model. This "brain" of our vulnerability scanner is now conceptually complete.

Today, we shift our focus from the control plane to the data plane—the distributed workers that perform the actual scans. We'll begin by diving into the most fundamental capability of a network scanner. This lesson addresses the learning outcome: Compare SYN, connect, and UDP port scanning techniques, analyzing their network mechanisms and stealth trade-offs. By the end of this lesson, you'll have a first-principles understanding of how these core scanning techniques work at the packet level.

1. The Foundation: TCP and UDP Protocols

Before we can analyze scanning techniques, we must solidify our understanding of the underlying network protocols they manipulate: TCP and UDP. As a software engineer, you're likely familiar with these, but a quick review from a security perspective is crucial. The differences between them are the very reason different scanning techniques are necessary.

To summarize the key points for our context:

• TCP (Transmission Control Protocol): A reliable, connection-oriented protocol. It establishes a formal connection (the three-way handshake) before data is exchanged, guaranteeing that packets arrive in order. This handshake is the mechanism we will exploit.
• UDP (User Datagram Protocol): A connectionless protocol. It's a "fire-and-forget" system with no guarantee of delivery or order. This lack of a formal handshake makes scanning for UDP services fundamentally different and more challenging.

2. The Straightforward Approach: TCP Connect Scan (-sT)

The TCP Connect scan is the most basic and reliable method for TCP port scanning. It behaves just like any standard network application, such as your web browser connecting to a server.

From your perspective as a developer, this scan works by asking the operating system to establish a full connection to a target port using the connect() system call.

• If the connect() call succeeds, the port is open. The scanner then immediately closes the connection.
• If the connect() call fails (e.g., with a "Connection refused" or timeout error), the port is closed or filtered.

To see this in action, the following video demonstrates the scan and, more importantly, shows the resulting network traffic.

Connect Scan Trade-offs:

• Pro: Highly reliable because it uses the OS's standard networking stack.
• Pro: Does not require special (root) privileges, as any user can make a connect() call.
• Con: Not stealthy. Because a full connection is established, the target application will be aware of the connection and will likely log it (e.g., an entry in syslog). This makes the scan easy to detect.
• Con: Relatively slow due to the overhead of completing the handshake and tearing down the connection for every single port.

3. The Stealthy Alternative: TCP SYN Scan (-sS)

The SYN scan, often called a "half-open" or "stealth" scan, is the default and most popular scan type used by Nmap. Its goal is to determine if a port is open without completing the TCP three-way handshake. This requires the ability to craft and send raw IP packets, bypassing the OS's connect() call. This is why it requires elevated privileges.

Here's the mechanism:

1. The scanner sends a SYN packet, as if it's about to start a real connection.
2. It then analyzes the response:
   • If it receives a SYN/ACK packet, the port is open. The target is willing to establish a connection. Instead of sending the final ACK to complete the handshake, the scanner sends a RST packet, tearing down the connection before it's fully established and logged by the application layer.
   • If it receives a RST packet, the port is closed.
   • If no response is received, the port is considered filtered (likely blocked by a firewall).

Now, let's watch the video demonstration and Wireshark capture, which will clearly contrast with the Connect scan.

SYN Scan Trade-offs:

• Pro: Much stealthier than a Connect scan. Since the connection is never completed, many applications and logging systems will not record the attempt. Modern Intrusion Detection Systems (IDS) can still detect it, however.
• Pro: Faster than a Connect scan because it doesn't have the overhead of completing the handshake.
• Con: Requires elevated (root or administrator) privileges to create the necessary raw packets.

4. The Challenge of Statelessness: UDP Scan (-sU)

Scanning for UDP services is fundamentally different and more difficult. Since UDP is connectionless, there is no handshake to exploit. Instead, the strategy relies on eliciting a specific response—or interpreting the lack of one.

Here is the general mechanism:

1. The scanner sends a UDP packet to the target port. (For common services like DNS, this packet might contain a valid probe; otherwise, it's often empty).
2. The response determines the port state:
   • If any UDP response is received, the port is open.
   • If an ICMP "Port Unreachable" error (Type 3, Code 3) is returned, the port is closed. The target host's OS is explicitly telling you nothing is listening there.
   • If no response is received after several retransmissions, the port is classified as open|filtered. It could be open (and the service just doesn't respond to the probe), or a firewall could be silently dropping the packet. There is no way to know for sure.
   • If another ICMP error is returned (e.g., "Host Unreachable"), the port is filtered.

UDP Scan Trade-offs:

• Pro: It's the only way to discover services running over UDP, which includes critical infrastructure services like DNS and DHCP.
• Con: Extremely slow. The reliance on timeouts for open|filtered ports drastically increases scan time.
• Con: Unreliable. UDP packets can be lost in transit without any notification.
• Con: ICMP error responses, which are needed to identify closed ports, are often rate-limited by modern operating systems, forcing the scanner to slow down even further.

5. Summary and Comparison

Choosing a scan technique involves balancing the need for information with constraints on speed, stealth, and privilege. Let's summarize the key differences in a table.

Feature	TCP Connect Scan (-sT)	TCP SYN Scan (-sS)	UDP Scan (-sU)
Network Mechanism	Full OS connect() handshake	Raw packet "half-open" handshake	Send UDP packet, analyze response/ICMP error
Privileges Needed	No	Yes (root/administrator)	Yes (for raw packets and ICMP)
Speed	Slow	Fast	Very Slow
Stealth	Low (logged by applications)	High (avoids application logs)	Medium (less activity, but timeouts are noisy)
"Open" Indication	Connection success	SYN/ACK response	Any UDP response
"Closed" Indication	Connection failure	RST response	ICMP Port Unreachable
Ambiguity	Low	Low	High (`open

From an architectural standpoint for the scan worker you're designing, this means the engine must be able to:

• Select between using a high-level OS socket API (connect) or a low-level raw socket interface depending on the scan type and available privileges.
• Parse low-level packet headers (TCP flags, ICMP types and codes) to interpret scan results correctly.
• Manage a complex state machine, especially for UDP scans, which involves aggressive timeout and retransmission logic.

Conclusion

In this lesson, we dissected three fundamental port scanning techniques at the network level. You now have a first-principles understanding of how they exploit or work around the core behaviors of the TCP and UDP protocols.

Key Takeaways:

• TCP Connect Scan is reliable and simple but slow and easily detected. It uses the standard OS connect() system call.
• TCP SYN Scan is fast and stealthy, avoiding application-level logs by never completing the three-way handshake. It requires raw packet privileges.
• UDP Scan is necessary but challenging due to the connectionless nature of UDP. It is slow, less reliable, and often results in an ambiguous open|filtered state.

Preview of the Next Lesson:

We've explored what these scanning techniques are. The next logical step is to consider how to implement them efficiently. In our next lesson, we will begin architecting the worker itself: "Design a modular port scanning engine with parallelism, state management, and timeout handling." We will take the theoretical techniques from today and start building the software framework to execute them at scale.