Designing Policy Engine Evaluation Logic

Hello! Welcome back to our course on designing security products.

In our last lesson, we architected a robust resource discovery system. We now have a mechanism that uses a hybrid of event-driven updates and periodic polling to maintain a near real-time, normalized inventory of our multi-cloud assets. This inventory, likely stored in a graph database, is our "source of truth."

But data is only useful when you can ask questions of it. This brings us to the core of our CSPM's functionality. Now that we know what resources exist and how they're configured, we need to determine if that configuration is secure.

This leads us to today's learning outcome: Design the core evaluation logic of a policy engine that runs rules against discovered cloud resource configurations.

We'll dissect the concept of a policy engine, starting from first principles. We'll explore how to represent security policies as code and design the engine that executes these policies against our cloud inventory to find misconfigurations.

---

1. What is a Policy Engine? A First-Principles View

At its heart, a policy engine is a system that separates decision-making logic (the "policy") from the application's main business logic. In our CSPM, the "application" is the data pipeline that collects and normalizes resource data. The "policy engine" is the component that evaluates this data against a set of security rules.

Instead of hard-coding thousands of if/else statements in our application, we offload this logic to a specialized engine. Let's see how this transformation looks in practice.

As the video illustrates, the most basic rules engine has three components:

1. State: The data to be evaluated. In our case, this is a cloud resource's configuration from our inventory.
2. Rules: A collection of (condition, action) pairs.
   • Condition: A function that takes the state and returns true or false (e.g., is_s3_bucket_public(resource)).
   • Action: A function that executes if the condition is true (e.g., create_finding("S3 bucket is public")).
3. Engine: An evaluator that iterates over the rules, applies their conditions to the state, and triggers the corresponding actions.

For a CSPM, we want to find all violations, not just the first one. This maps to the "run all" concept mentioned later in the same video (17:07 - 18:27), where every rule whose condition is met gets its action executed.

---

2. From Simple Rules to Declarative Policies

While the simple rules engine is a great starting point, production-grade policy systems often use a more powerful, declarative approach. Instead of writing imperative code (if this, do that), you declare the properties of a desired state (the bucket must not be public).

The industry standard for this is the Open Policy Agent (OPA) and its declarative language, Rego. OPA provides a general-purpose policy engine that we can adapt perfectly for our CSPM.

The architectural model of OPA is simple but powerful:

• Policy: A set of rules written in Rego.
• Data: The JSON document to evaluate the policy against. For us, this is the configuration of one or more cloud resources from our inventory.
• Query: A question we ask the engine, like "which deny rules are violated for this input?"
• Decision: The JSON output from the engine, e.g., a list of violation messages.

Let's get a deeper look at this architecture and the Rego language.

This declarative model is ideal for a CSPM because security policies can be written and managed by security analysts without needing to change the core Go/Python/Java code of our data collection pipeline.

---

3. Architecting the Evaluation Logic

Now let's dive into the core of the design. How does an engine like OPA actually evaluate a set of declarative rules against a large dataset of cloud configurations? It's not a simple loop.

Evaluation as a Proof Tree

Declarative languages like Rego don't execute linearly. The engine's job is to find a way to prove that a query is true based on the available rules and data. This process is best visualized as the traversal of a proof tree.

The core insight is that evaluation alternates between two types of logical operations:

• OR nodes: These represent the different rules that could satisfy a query. If you have three different deny rules for S3 buckets, the engine can satisfy the query deny by proving any one of them is true.
• AND nodes: These represent the different conditions within a single rule. For a rule to be true, all of its conditions must be true simultaneously.

Let's consider a practical CSPM example in a Rego-like syntax:

# A rule is satisfied if ALL conditions in its body are true (AND)
deny[msg] {
    bucket := input.resources[_]         # Select a resource
    bucket.type == "aws_s3_bucket"
    bucket.attributes.acl == "public-read"
    msg := sprintf("S3 bucket %s is public", [bucket.name])
}




# The overall query for 'deny' is satisfied if this rule OR another 'deny' rule is true
deny[msg] {
    bucket := input.resources[_]
    bucket.type == "aws_s3_bucket"
    not bucket.attributes.server_side_encryption_configuration
    msg := sprintf("S3 bucket %s lacks encryption", [bucket.name])
}

The engine's evaluation process is to traverse this AND/OR tree to find all combinations that result in a deny message.

Two-Stage Evaluation for Efficiency

Our cloud inventory could contain millions of resources. Evaluating every rule against every resource would be incredibly inefficient. A more sophisticated architecture uses a two-stage evaluation process, an idea well-articulated in database and constraint-solving research.

While the full academic paper LINK ("Scaling a Declarative Cluster Manager Architecture...") is quite dense, its core architectural idea is highly relevant:

1. Relation Evaluation (Data Fetching): Before evaluating the full logic, the system first performs a targeted query against its database to fetch only the data relevant to the policy. For a policy about S3 encryption, this stage would query our graph database for all aws_s3_bucket resources and their associated encryption configurations.

2. Constraint Evaluation (Logic Execution): The policy engine's core logic then runs against this much smaller, pre-filtered dataset.

This pattern drastically improves performance by ensuring the computationally expensive logic evaluation only operates on the necessary data.

Designing for Performance: Lazy Evaluation

Real-world policies can be complex and recursive (e.g., tracing IAM permissions through multiple groups and roles). If the engine tried to compute the entire proof tree upfront, it could lead to infinite loops or out-of-memory errors.

A critical design choice for the core evaluation logic is to use lazy evaluation. As highlighted in the Oso blog post you read, the proof tree is not generated all at once. Instead, it's expanded one level at a time, on demand. This allows the engine to handle very complex or even infinitely recursive policies efficiently by only exploring the necessary branches of the proof tree.

---

4. A High-Level Design for the CSPM Policy Engine

Let's synthesize these concepts into a high-level architecture for our CSPM's policy engine.

Our engine would consist of the following logical components:

1. Policy Loader: A component responsible for loading and parsing all the Rego policy files from a repository. It provides the parsed rules to the evaluator.

2. Evaluation Orchestrator: This component manages the overall evaluation process. When a scan is triggered (either for a single resource update or a full-inventory scan), it does the following:
   a. Determines which policies are applicable (e.g., AWS policies for AWS resources).
   b. Coordinates with the Data Fetcher and the Core Evaluator.

3. Data Fetcher (Query Planner):

   • Receives a request from the Orchestrator with the target resource(s) and policies.
   • Analyzes the policies to understand their data requirements.
   • Queries the main resource inventory (our graph database) to pull the specific subset of data needed for the evaluation.
   • Passes this data subset as the input document to the Core Evaluator.

4. Core Evaluator (The "Brains"):

   • An embedded OPA library or a custom-built engine based on the principles we've discussed.
   • Receives the policies from the Policy Loader and the input data from the Data Fetcher.
   • Executes the query (e.g., data.cspm.deny) by lazily traversing the AND/OR proof tree.
   • Returns the result—a structured JSON object containing the set of all generated deny messages.

5. Findings Processor:

   • Receives the raw output from the Core Evaluator.
   • Transforms this output into the canonical "finding" format for our CSPM.
   • Routes the findings to a database, an alerting system, or a user interface.

This modular design decouples data fetching from logic evaluation, enabling optimization and scalability.

---

Conclusion

In this lesson, we designed the analytical core of our CSPM. We've moved beyond simply collecting data to actively interpreting it to find security risks.

Key Takeaways:

• Decoupling is Key: A policy engine separates what to check (the policy) from how to check it (the engine), making the system more flexible and maintainable.
• Declarative is Powerful: Languages like Rego allow policies to be expressed as statements of fact about a desired state, which is a natural fit for security and compliance rules.
• Evaluation is a Tree Traversal: The core evaluation logic is not a simple loop but a process of proving a query by traversing a branching proof tree of AND and OR conditions.
• Efficiency Through Architecture: A scalable design uses a two-stage process of first fetching relevant data and then running the logic engine on that smaller subset. Techniques like lazy evaluation are crucial for handling complex, recursive policies.

Preview of the Next Lesson

We've designed an engine that can run policies. But where do these policies come from? How are they written, tested, and managed? And how do we ensure they align with industry standards?

In our next lesson, we will architect a rule authoring subsystem and a process for mapping rules to external benchmarks like CIS. We will design the tools and workflows that security analysts use to build and maintain the intelligence that powers our entire CSPM platform.