DDoS Mitigation with BGP Anycast and Distributed Scrubbing Centers

Hello! Welcome to the third module of our course.

In the previous module, we completed our deep dive into the architecture of a Web Application Firewall (WAF). Our final lesson focused on designing a stateful session tracking system, enabling the WAF to correlate multiple requests and detect complex, low-and-slow attacks. We've built a solid understanding of how to protect a single application (or a set of applications) at Layer 7.

Now, we're shifting our focus to a different class of threat that operates at a much larger scale: Distributed Denial-of-Service (DDoS) attacks. While a WAF is excellent at stopping sophisticated application-layer attacks, it's often powerless against volumetric attacks that aim to saturate the network infrastructure long before traffic even reaches the application server.

This brings us to today's learning outcome: Architect a global traffic ingestion system for DDoS mitigation using BGP anycast and distributed scrubbing centers. We will design the first and most critical layer of a modern DDoS protection service—the global network that can absorb and filter continent-spanning attacks.

1. The Challenge: Beyond Application Logic

A volumetric DDoS attack doesn't try to be clever. It's a brute-force assault. The goal is simple: send so much junk traffic to a target's IP address that its internet connection becomes completely saturated. Imagine trying to drink from a firehose; it doesn't matter how sophisticated your digestive system is if the sheer volume overwhelms you.

This type of attack targets Layers 3 (Network) and 4 (Transport) of the OSI model.

A single data center, even with terabits of capacity, represents a single chokepoint. To defend against a global botnet, you need a globally distributed defense.

2. Distributing the Load: Unicast vs. Anycast

Before we design the system, we need to understand the fundamental routing technology that makes global DDoS mitigation possible. Your background in distributed systems will be useful here as we compare different ways to route traffic to multiple locations.

A traditional service hosted in one data center uses Unicast addressing. A DNS name resolves to a single IP address, and all traffic goes to that one location. This is simple but fragile.

What if we have multiple data centers? Two common approaches for distributing traffic are DNS-based load balancing (GSLB) and IP-based routing (Anycast).

As the video explains, Anycast is the foundational technology for a global DDoS ingestion network. By announcing the same IP prefix from dozens or hundreds of locations simultaneously, you effectively turn your network into a single, massive, distributed entity. An attacker trying to flood your service IP is no longer aiming at a single data center; they're aiming at your entire global network. The attack traffic is naturally absorbed by the Points of Presence (PoPs) closest to the attack sources, distributing the load across your infrastructure.

3. The Architectural Blueprint: Divert, Scrub, Return

Now that we have our traffic distribution mechanism (Anycast), we need to design the facilities that will actually handle the traffic. This is where scrubbing centers come in.

A scrubbing center is a specialized data center with two key characteristics:

1. Massive Bandwidth: Capable of absorbing terabits per second of traffic.
2. Specialized Equipment: High-performance hardware and software for filtering malicious traffic at line rate.

These scrubbing centers are the nodes in our global Anycast network. The overall architecture follows a clear pattern.

Let's synthesize this into a clear architectural flow:

Step 1: Onboarding & BGP Announcement (Peacetime)
To protect a customer, the DDoS mitigation provider takes over the advertisement of their IP prefixes. Instead of the customer announcing 198.51.100.0/24 from their single data center, the provider announces it via BGP from all of their global scrubbing centers using Anycast.

Step 2: Attack Detection and Diversion
When an attack is detected (we'll cover detection in the next lesson), the system may need to change how traffic is routed. While Anycast naturally distributes traffic, sometimes an operator needs to manually redirect a flood targeting one PoP to other PoPs with more capacity. This is often done using BGP diversion. The provider updates its BGP announcements to make the path to the overloaded PoP less attractive, causing internet routers to redirect traffic to other scrubbing centers.

Step 3: Scrubbing
Inside the scrubbing center, traffic undergoes multi-layer inspection. For the volumetric attacks we're focused on today, this involves:

• Stateless packet filtering (e.g., dropping malformed packets).
• Rate limiting per source IP.
• Connection validation (e.g., SYN cookies to defeat SYN floods).

The goal is to discard the vast majority of junk traffic as efficiently as possible.

Step 4: Returning Clean Traffic
The legitimate traffic that survives the scrubbing process must be sent back to the customer's actual application servers. This is typically done using an IP tunneling protocol like GRE (Generic Routing Encapsulation). The scrubbing center encapsulates the clean packets inside a new IP packet and sends them over the public internet to the customer's edge router, which unwraps them and forwards them to the origin servers.

4. A Deeper Dive: Active Traffic Engineering with BGP

For a system architect, simply saying "BGP does it" isn't enough. Your "first principles" mindset demands we look deeper. How can an operator actively control this massive, distributed system during an attack?

DDoS providers don't just rely on default BGP behavior. They use a set of traffic engineering techniques to manipulate routing paths in real-time. This is often guided by a pre-computed "playbook."

From the paper, we can extract the primary tools used to build the control plane for our traffic ingestion system:

1. AS-Path Prepending: BGP prefers paths with a shorter AS-Path (fewer network hops). To make a route through a specific scrubbing center less attractive, the operator can artificially lengthen its AS-Path by "prepending" its own AS number multiple times. This encourages traffic to flow to other centers with shorter, more preferable paths.

   • Use Case: Shift load away from an overloaded PoP.

2. BGP Communities: These are tags that can be attached to a BGP route announcement. While not part of the core path selection algorithm, upstream providers (like major ISPs) can be configured to interpret these tags and apply specific policies. For example, a provider might honor a community tag that says "do not announce this route to peers in Europe."

   • Use Case: More granular, policy-based control, such as isolating an attack source to a specific region.

3. Path Poisoning: This is a more aggressive technique where you include a target network's AS number in your AS-Path announcement to them. This makes the route look like a loop to that network, causing them to reject it entirely.

   • Use Case: Completely block a route from a specific peer network that is sourcing a large amount of attack traffic.

By mapping out the effects of these changes beforehand, a DDoS provider builds a BGP Playbook. When an attack hits, the operator or an automated system can quickly select the appropriate "play" to shift traffic, isolate the attack, and keep the service online.

Conclusion

In this lesson, we have architected the massive front door for a DDoS mitigation service. We've moved from a fragile, single-site model to a resilient, globally distributed architecture capable of absorbing enormous traffic volumes.

Key Takeaways:

• Global Ingestion is Key: To fight a distributed attack, you need a distributed defense. A global network of scrubbing centers is the standard architecture.
• Anycast for Distribution: BGP Anycast is the core technology used to distribute incoming traffic across this global network, preventing any single point from being overwhelmed.
• The Flow: Announce, Divert, Scrub, Return: The process involves the provider announcing the customer's IPs, using BGP to direct traffic to scrubbing centers, filtering out malicious packets, and tunneling clean traffic back to the origin.
• BGP is the Control Plane: Advanced traffic engineering using BGP mechanisms like AS-Path prepending and communities allows operators to actively manage traffic flow during an attack, guided by a pre-computed playbook.

Preview of the Next Lesson:

We have successfully designed a system to ingest and route massive amounts of traffic. But this system is currently blind. It doesn't know an attack is happening until an operator tells it. In our next lesson, we will address this by designing the brain of the operation: Design detection mechanisms for common Layer 3/4 DDoS attack vectors such as SYN and UDP floods.