Building a Cumulative Anomaly Risk Score Engine

Hello! Welcome back to our course on architecting security products.

In our previous lessons, we designed the core of a WAF's inspection pipeline. We started with a modular design for chaining security modules, then built a high-performance signature engine using Aho-Corasick, and finally explored advanced optimization techniques like caching, JIT compilation, and hardware offload. So far, our engine makes a binary decision: a request either matches a malicious signature and is blocked, or it doesn't.

This approach is powerful but can be brittle. What if an attacker sends a request that isn't definitively malicious but exhibits several suspicious characteristics? A single rule might not be enough to block it, but the combination of signals suggests a high risk.

This brings us to today's learning outcome: Design an anomaly scoring engine that combines multiple weak signals into a cumulative risk score for a request. We will move beyond simple block/allow logic and architect a system that aggregates evidence to make more nuanced and accurate security decisions.

1. The Core Concept: From Binary Decisions to Anomaly Scoring

The fundamental idea behind anomaly scoring is to separate detection from blocking. Instead of a single rule making a final decision, each rule that triggers contributes "points" to a cumulative score for the request. The final action (e.g., block, log, or allow) is then based on whether this total score exceeds a predefined threshold.

This model is a cornerstone of modern WAFs, most famously in the OWASP ModSecurity Core Rule Set (CRS). Let's watch a brief explanation of how it works in practice.

As the video explains, this approach offers two major advantages:

• Resilience to False Positives: A single, slightly sensitive rule flagging a legitimate request won't cause an immediate block. The request is only blocked if it accumulates a high enough score from multiple indicators.
• Detection of Stealthy Attacks: It can catch sophisticated attacks that are designed to fly under the radar. An attacker might try several low-impact probing techniques in a single request. While each individual technique might only add a few points to the score, their sum can reveal the malicious intent.

Now, let's architect an engine that can implement this logic.

2. Architecting the Scoring Engine

Designing a risk assessment or anomaly scoring engine involves a systematic process of identifying signals, quantifying risk, and making decisions. We can break this down into several key components.

Based on this framework, let's design our engine's architecture.

Component 1: Signal Generation (Identifying Risk Factors)

First, we need the "weak signals" that will serve as inputs. These are any observable attributes of a request that can indicate risk. In our WAF architecture, these signals come from various inspection modules that run within our modular pipeline:

• Signature Engine: A match for a specific pattern (e.g., ../ for path traversal) is a strong signal.
• Protocol Conformance Engine: Violations of RFC standards (e.g., invalid characters in headers, incorrect Content-Length) are signals.
• IP Reputation Module: The source IP's reputation score (e.g., known TOR exit node, history of abuse) is a signal.
• HTTP Header Sanity Check: The presence of unexpected or missing headers (e.g., a missing User-Agent) can be a signal.

The image below illustrates how a commercial WAF gathers signals from various sources to detect anomalies.

Component 2: Score Aggregation (Quantifying Risk)

This is the core of the engine. Once signals are generated, we need to combine them into a single score. There are a few models for this, with increasing complexity:

Model A: Simple Summation
This is the model used by the OWASP CRS, as seen in the video. Each triggered rule adds a predefined number of points to a running total. The points are typically categorized by severity.

• Critical Risk Rule: +5 points
• High Risk Rule: +4 points
• Medium Risk Rule: +3 points
• Low Risk Rule: +2 points

Example:
A request triggers one "Critical" rule and two "Low" risk rules.
Total Score = $5+2+2=9$

Model B: Weighted Summation
A more flexible approach assigns a weight to each category of signal, reflecting its relative importance. This allows you to tune the engine's sensitivity to different types of threats.

The final score is calculated as:

Let's use the practical example from the Tencent Cloud article (LINK, Section "Example Workflow in Practice") to see this in action:

• Risk Factor: High transaction amount -> Signal Score: 35, Weight: 40%
• Risk Factor: New device -> Signal Score: 25, Weight: 30%
• Risk Factor: Unusual location -> Signal Score: 30, Weight: 30%

Total Score = $(0.40\times 35)+(0.30\times 25)+(0.30\times 30)=14+7.5+9=30.5$
(Note: The article simplifies this to a sum, but a weighted approach provides more control).

The weights and scores are determined by security experts based on empirical data and the specific risk profile of the application being protected.

Component 3: Decision Logic

The final step is to use the cumulative score to make a decision. A simple block/allow threshold is a start, but a tiered approach provides more granular control.

• Score 0-4 (Low): ALLOW - The request is considered safe.
• Score 5-19 (Medium): LOG - Allow the request, but log it with full details for potential offline analysis. This helps in tuning and threat hunting.
• Score 20-49 (High): BLOCK - Deny the request and return a generic error page. Generate a security event log.
• Score 50+ (Critical): BLOCK_AND_ALERT - Block the request and trigger a high-priority alert to a Security Operations Center (SOC) for immediate investigation.

This multi-tiered logic allows the system to respond proportionally to the perceived level of risk.

3. Advanced Architecture: Multi-Stage Scoring

The models we've discussed assume that all signals can be generated and evaluated quickly. However, some of the most effective signals, particularly those derived from machine learning, can be computationally expensive. Running them on every single request would introduce unacceptable latency.

As a software engineer, you'll recognize this as a classic optimization problem. The solution is a multi-stage or tiered architecture, where we use cheap checks to filter the majority of traffic and reserve expensive analysis for only the most ambiguous cases. The domain of AI-based fraud detection provides a perfect analogy for this design pattern.

We can apply this exact architecture to our anomaly scoring engine:

1. Stage 1: High-Speed Triage Engine (Processes 100% of requests)

   • This is our weighted-summation engine. It's extremely fast, relying on signals from regex matching, IP lists, and other simple checks.
   • It makes immediate decisions for the vast majority of traffic:
     • Score < 5: ALLOW (clearly benign).
     • Score > 50: BLOCK (clearly malicious).

2. Stage 2: Deep Inspection Engine (Processes ~5% of requests)

   • Requests that land in the "grey area" (e.g., score between 5 and 49) are passed to this second stage.
   • This engine can be much more sophisticated and computationally intensive. It might be:
     • A gradient boosting model that analyzes hundreds of features engineered from the request and user session.
     • A behavioral analysis model that compares the user's current actions to their historical baseline.
     • As seen in the video, even an LLM could be used to analyze unstructured text in request bodies for subtle signs of social engineering or policy violation.
   • The output of this engine is a more confident risk score, which is then used for the final decision.

This multi-stage design provides the best of both worlds: low latency for the majority of traffic and high-accuracy detection for the tricky, ambiguous cases.

4. The Feedback Loop: Tuning and Adaptability

An anomaly scoring engine is not a static system. It must be tuned and adapted over time to be effective. An untuned engine can be crippled by false positives, rendering it useless.

Designing for tunability requires one critical architectural component: a feedback loop.

The key architectural takeaways for enabling this feedback loop are:

1. Log Everything: The engine must be designed to log the final anomaly score and all contributing signals for every single request, not just the ones that are blocked. Without data on benign traffic, it's impossible to understand the score distribution and identify false positives effectively.
2. Run in Non-Blocking Mode: The system must support a "logging-only" or "monitoring" mode. This allows the engine to be deployed with a low, aggressive threshold to gather data on what would be blocked, without actually impacting users.
3. Iterative Tuning Process: The architecture must support the operational workflow described in the article:
   • Start with a very high blocking threshold (e.g., 100) or in monitoring mode.
   • Collect traffic data for a period.
   • Analyze the score distribution to find the legitimate requests that are receiving high scores.
   • Adjust the scores/weights of the specific rules causing these false positives.
   • Incrementally lower the blocking threshold (e.g., from 100 to 50, then to 20, and finally to a production level like 5).
   • Repeat the process.

For ML-based engines, this feedback loop is even more direct. When a security analyst investigates an alert and marks it as a "False Positive" or a "True Positive", this labeled data is fed back into the system to retrain and improve the model over time.

Conclusion

Today, we designed a crucial component that adds intelligence and resilience to our WAF. By moving from simple binary rules to a cumulative scoring system, we created a more robust and tunable defense mechanism.

Key Takeaways:

• Anomaly scoring separates detection from blocking, allowing the system to aggregate multiple "weak signals" into a confident decision.
• A basic scoring engine uses weighted summation of signals from various inspection modules. The final score is compared against tiered thresholds to determine an action (e.g., allow, log, block).
• Advanced architectures use a multi-stage pipeline, where a fast, simple engine triages traffic and escalates ambiguous cases to a more powerful but computationally expensive ML-based engine.
• Designing for a feedback loop is critical. The system must log scores for all requests and support an iterative tuning process to manage false positives and adapt to changing traffic patterns.

Preview of Next Lesson:

Our anomaly scoring engine is excellent at evaluating the risk of a single, self-contained request. But what about attacks that unfold over time, across multiple requests? For example, a slow-rate credential stuffing attack or a user attempting to abuse business logic by probing an application's functions. To detect these, we need to maintain state. In our next lesson, we will design a session tracking system for stateful WAF inspection and multi-request attack correlation.