DDoS Mitigation Strategies & Auto-Scaling Design

Hello! Welcome back to our course on architecting security products.

In our previous lessons, we designed a complete data path for a DDoS scrubbing system, from traffic ingestion with BGP Anycast to returning cleaned traffic to origin servers using methods like GRE tunneling or the PROXY protocol. We have a functional scrubbing pipeline. The next critical architectural question is: when should this pipeline be used?

This brings us to this lesson's learning outcome: to evaluate the trade-offs between always-on vs. on-demand DDoS mitigation and design reactive auto-scaling mechanisms. We will explore two fundamentally different philosophies for engaging a DDoS defense system and then dive deep into the design of the "reactive" components that make an on-demand strategy possible.

1. The Core Architectural Choice: Always-On vs. On-Demand

At the highest level, you have two choices for integrating a DDoS scrubbing service:

1. Always-On: All your traffic, legitimate and malicious alike, is permanently routed through the scrubbing provider's network. Your BGP announcements always direct traffic to the scrubbing centers.
2. On-Demand (or Reactive): Your traffic flows directly to your infrastructure during "peacetime." Only when a sufficiently large attack is detected is traffic diverted to the scrubbing provider for filtering.

This choice is one of the most fundamental trade-offs in DDoS defense architecture, balancing reaction time against cost, latency, and control.

Let's break down the trade-offs based on the article and our architectural principles:

For many organizations, the ideal is the DEFCON 2 principle mentioned in the FastNetMon article: design a system that can react automatically and elevate its defense posture, rather than staying at high alert (Always-On) permanently. This leads us to the next question: how do we design that reactive system?

2. Designing Reactive Mitigation Mechanisms

An effective on-demand system has two key parts: a detection mechanism that generates a signal, and an automated response mechanism that acts on that signal. We will explore two primary forms of reaction: traffic diversion to an external scrubber and infrastructure auto-scaling to absorb the load.

A. Reaction via Traffic Diversion (BGP Automation)

When a large-scale attack is detected, the goal is to reroute traffic to your scrubbing provider. The de facto tool for this on the internet is the Border Gateway Protocol (BGP).

However, the way you use BGP matters immensely. A naive approach can be worse than doing nothing.

The Anti-Pattern: Remote Triggered Black Hole (RTBH)
A common, but crude, mechanism is RTBH. Upon detecting an attack on an IP prefix, your system automatically announces to your upstream providers to drop all traffic destined for that prefix.

As the author argues, RTBH is "voluntary extinction." You take your own service offline to "protect" it, which is a losing proposition. It's a triage tool for preventing catastrophic collateral damage, not a defense strategy.

The Correct Design: BGP-Triggered Scrubbing
A proper design uses BGP not to drop traffic, but to selectively divert it. The architecture looks like this:

1. Detection: Your monitoring system (e.g., using NetFlow, sFlow, or application metrics) detects an attack exceeding a certain threshold.
2. Automation: The detection system sends a signal to an automation controller.
3. Activation: The controller instructs your edge routers to change their BGP announcement for the attacked prefix. This is typically done by adding a specific BGP community to the announcement sent towards your scrubbing provider.
4. Scrubbing: The scrubbing provider's network is configured to recognize this community. When it sees an announcement with this tag, it accepts the route, draws the traffic for that prefix into its network, scrubs it, and forwards the clean traffic to you (using one of the mechanisms from our last lesson, like a GRE tunnel).

This flow—Detection → Automation → BGP Community Tag → Scrubber Activation—is the core of a modern, on-demand DDoS mitigation platform. It allows for surgical, automated responses that preserve service availability. More advanced versions of this concept, like BGP FlowSpec, allow you to push specific filtering rules (e.g., "drop UDP traffic on port 53 from source X") to your routers, offering even more granularity.

B. Reaction via Infrastructure Auto-Scaling

Not every attack requires a full-scale diversion to a scrubbing center. For lower-volume application-layer (L7) attacks, a well-designed system can use cloud auto-scaling to absorb the malicious traffic and maintain service. The idea is simple: if CPU or request count goes up, add more servers.

However, an intelligent attacker can turn this defense into a weapon against you. This leads to what's known as an Economic Denial of Sustainability (EDoS) or a Yo-Yo Attack.

As the paper describes, auto-scaling relies on a set of rules and timers. The Yo-Yo attack is designed to exploit the delays inherent in this system:

1. Attacker sends a burst of traffic, triggering a scale-up alert.
2. The system waits for the Iup interval. Load is high, performance degrades.
3. The system provisions new machines. This takes Wup (warming time), during which you're paying for the machines but they aren't serving traffic yet. Performance is still degraded.
4. Just as the new machines come online and the load stabilizes, the attacker stops the attack.
5. The system now sees low load. It waits for the Idown interval. During this time, you are paying for idle, over-provisioned capacity (EDoS).
6. The system scales down. As soon as the machines are de-provisioned, the attacker starts the attack again, returning to step 1.

The system is caught in a "yo-yo" of scaling up and down, causing both persistent performance degradation and inflated costs.

3. Designing a Resilient Reactive Auto-Scaler

The Yo-Yo attack demonstrates that a naive auto-scaling configuration is a liability. To design a more resilient system, we must tune our reactive mechanisms to be less predictable and more robust.

Based on the analysis in the paper LINK, here are our key design levers and trade-offs:

1. "Scale Up Quickly, Scale Down Slowly": This is the most critical defense. By setting a long scale-down interval (Idown), you prevent the system from quickly removing capacity when the attacker enters an "off" phase.

   • Trade-off: You are explicitly choosing to accept higher cloud costs (potential economic damage) to prevent performance degradation from subsequent attack waves. You are banking on the assumption that an attack lull is temporary.

2. Minimize Warming Time (Wup): The Wup is pure pain—you're paying for resources that aren't helping. Reducing it is a huge win.

   • Design Tactics: Use pre-configured "golden" machine images, leverage lightweight containers (e.g., Docker, which starts much faster than a full VM), and optimize your application's bootstrap sequence.

3. Choose the Right Scaling Policy: The paper's analysis shows that while an adaptive policy seems more advanced, it's more vulnerable to the Yo-Yo attack because it scales up all machines at once, maximizing the cost during the Wup and subsequent "off" phase. A discrete (e.g., "add one instance at a time") policy is slower to react but suffers less damage from this specific attack profile.

   • Trade-off: A discrete policy might be too slow to handle a legitimate flash crowd. The choice depends on your expected traffic patterns and risk tolerance.

4. Set Limits: To prevent catastrophic EDoS, always configure a maximum number of instances for your auto-scaling group.

   • Trade-off: You are capping your ability to scale. This cap must be high enough to handle legitimate peak traffic but low enough to prevent a runaway bill from an attack.

Conclusion

In this lesson, we dissected the critical architectural decision between always-on and on-demand DDoS mitigation and designed the core components of a reactive defense system.

Key Takeaways:

• Always-on vs. On-demand is a trade-off between the zero-second mitigation time of always-on and the lower cost, reduced latency, and greater control of an on-demand approach.
• Reactive traffic diversion should be built on automated BGP announcements using communities to trigger a scrubbing service, not on self-destructive tools like RTBH.
• Infrastructure auto-scaling is a useful first line of defense but is vulnerable to Yo-Yo attacks that exploit its timing logic to create both performance and economic damage.
• Designing a resilient auto-scaler involves deliberately making it less "twitchy" by scaling down slowly, while also minimizing warm-up times and setting hard limits to contain costs.

Preview of the Next Lesson:

The most sophisticated DDoS attacks are often at Layer 7, mimicking legitimate user behavior. Detecting these attacks requires a deeper understanding of the client. This leads directly to our next module, "Architecting Bot Management Systems." In the first lesson, we will design a data collection pipeline for an IP reputation system using honeypots, threat intelligence feeds, and application logs, which is a foundational component for distinguishing human users from malicious bots.