Designing a Low-Latency Authorization Engine

Hello! Welcome back to our course on architecting security products.

In our previous lesson, we compared RBAC and ABAC policy models and designed their underlying data structures. We concluded that while ABAC offers incredible flexibility, its power comes with a significant challenge: evaluating potentially thousands of policies for every request without introducing crippling latency. We identified that the key to solving this is to pre-compile declarative policies into an optimized data structure, like a tree, for fast evaluation.

Today, we will build directly on that foundation to tackle our next learning outcome: Architect a low-latency policy evaluation engine for authorization, including caching and distribution strategies.

We will design the system that executes these policies. You'll see how your experience with distributed systems is directly applicable as we explore architectural patterns for deploying this engine and dive deep into the caching and replication strategies required to meet the stringent low-latency demands of real-time authorization.

1. The Core Components of an Authorization Service

Any authorization service, regardless of the policy model (RBAC, ABAC, etc.), can be broken down into a standard set of logical components. These concepts, formalized by NIST, provide a crucial separation of concerns for our design.

• Policy Enforcement Point (PEP): This is the gatekeeper. It's a component that intercepts a user's request within your application (e.g., as middleware in an API gateway). It doesn't make decisions itself; its job is to ask for permission.
• Policy Decision Point (PDP): This is our "engine." The PEP sends the request context to the PDP. The PDP contains the policy evaluation logic—it evaluates the relevant policies against the request attributes and returns a simple Allow or Deny decision.
• Policy Information Point (PIP): The PDP often needs more information than what's in the initial request. The PIP is responsible for fetching any necessary attributes from external sources, such as user roles from an employee database, resource metadata from the application's database, or threat intelligence data.
• Policy Administration Point (PAP): This is where policies are created, updated, and managed. The PAP feeds the policies to the PDP.

For the rest of this lesson, we will focus on designing the Policy Decision Point (PDP), as it is the heart of the evaluation engine.

2. Architectural Deployment Patterns for the PDP

Where does the PDP live in our system architecture? The choice has major implications for latency, scalability, and manageability. Let's examine three common patterns.

1. Embedded Library: The PDP is compiled directly into your application as a library. The PEP (your application code) makes a simple function call to the PDP.

   • Pros: The absolute lowest latency, as it's an in-process call. No network overhead.
   • Cons: Tightly coupled. Updating the policy engine logic requires rebuilding and redeploying the entire application. It's also language-specific.

2. Sidecar Container: The PDP runs as a separate process or container alongside the application on the same host (e.g., in a Kubernetes Pod). The PEP communicates with the PDP over the local loopback network interface (localhost).

   • Pros: Decoupled and language-agnostic. The PDP can be updated independently of the application.
   • Cons: Introduces a small amount of latency due to the network call, but it's minimal since it doesn't leave the host.

3. Centralized Service: The PDP is deployed as a standalone, centralized service. Multiple applications in your infrastructure make network calls to this shared authorization service.

   • Pros: Centralized policy management and easy to share across many services.
   • Cons: Highest latency, as it involves a full network round-trip. It also introduces a potential single point of failure and a scaling bottleneck if not designed carefully.

Given your background in distributed systems, you'll recognize this as a classic trade-off between performance and centralized control. For services where authorization is on the critical path of every single request, the sidecar and embedded library patterns are often preferred to minimize latency.

3. Achieving Low Latency: Caching Strategies

Authorization checks are high-frequency and repetitive. Caching is not an optimization; it's a fundamental requirement for a low-latency policy engine.

Now, let's apply these ideas to the ultimate case study in low-latency authorization: Google Zanzibar. Zanzibar is a global-scale authorization system that achieves single-digit millisecond latency. Understanding its caching strategies is key to designing our own high-performance engine.

The Zanzibar paper and associated talks reveal several sophisticated caching techniques that are crucial for its performance.

Let's break down the key caching strategies from the paper:

1. Multi-Level Distributed Cache: Zanzibar servers (aclservers) form a distributed cache for authorization decisions and intermediate results. They use consistent hashing on the object ID to deterministically map a request to a specific aclserver, maximizing cache locality. Results are even cached at both the caller and callee of internal RPCs, forming cache trees.

2. Cache Stampede Protection: When a popular, uncached item is requested simultaneously by many clients, it can overwhelm the backend database (a "cache stampede"). Zanzibar uses a lock table on each server. Only the first request for a given cache key proceeds to the database; others wait for the first request to populate the cache.

3. Timestamp Quantization: Authorization checks in Zanzibar can be evaluated at specific microsecond-resolution timestamps (zookies). This would destroy cache hit rates, as two requests for the same resource might have slightly different timestamps. Zanzibar's brilliant solution is to round up the evaluation timestamp to a coarser granularity (e.g., the nearest second). This dramatically increases the probability that concurrent requests will share the same cache key and hit the cache, without violating correctness.

4. Dynamic Hotspot Detection: For extremely popular objects, Zanzibar's aclservers will dynamically detect the high request volume and switch from looking up individual relationship tuples to reading and caching the entire set of relationships for that object. This trades a single, larger read for thousands of potential smaller lookups.

4. Achieving Scalability: Distribution Strategies

A low-latency engine that can't scale is useless. We must design our system to handle a massive volume of requests and data distributed globally. Again, we turn to Zanzibar for state-of-the-art solutions.

The core challenge is this: how do you perform a globally consistent authorization check without incurring a high-latency round trip to a central leader database on every request?

Zanzibar's distribution strategies are revolutionary:

1. Global Data Replication: All ACL data is replicated across dozens of data centers worldwide using Google Spanner. This ensures the data is physically close to the clients.

2. Snapshot Reads from Local Replicas (The "Zookie" Trick): This is the magic that enables low-latency local reads.

   • When content is modified, the application gets a zookie (an opaque timestamp) from Zanzibar. This zookie is stored alongside the content.
   • When a user tries to access that content, the application sends the content's zookie with the Check request.
   • Zanzibar can now evaluate the check at any snapshot at or after the zookie's timestamp. Since the data is replicated everywhere, there's a very high probability that a local replica is already up-to-date enough to satisfy the request without a cross-region call.
   • This masterfully avoids reading from a distant leader for the vast majority of checks, solving the latency problem.

3. Asynchronous Denormalization (Leopard Indexing): For very deep or wide permission graphs (e.g., checking if a user is in a group with thousands of nested subgroups), traversing the normalized graph is too slow. Zanzibar uses an offline system called Leopard to pre-calculate and flatten these relationships into a specialized index. Checks against these complex structures query the fast, denormalized Leopard index instead of the primary Spanner database. This is a classic read-optimization pattern where you accept higher write-path complexity for much faster read performance.

4. Request Hedging: To mitigate tail latency when talking to its dependencies (like Spanner or Leopard), Zanzibar sends a second, identical request if the first one doesn't respond within a certain percentile-based delay. Whichever returns first wins. This is a powerful technique for smoothing out performance in a distributed environment.

5. Policy Language as a Performance Feature

Finally, the design of the policy language itself is a critical architectural choice for a low-latency engine.

As the article highlights, designing for performance means making deliberate constraints in the policy language:

• No Loops or Recursion: Cedar's language explicitly forbids loops. This guarantees that policy evaluation will always terminate and does so in a bounded, predictable amount of time.
• Stateless Evaluation: Policies cannot change application state. This immutability allows evaluations to be safely parallelized and reasoned about independently.
• Policy Slicing: Cedar's structured principal, action, resource scope allows applications to perform a pre-filtering step. Instead of sending all 10,000 policies to the engine, the application can use the request's principal and resource to determine that only a small subset of policies could possibly apply. This "slicing" dramatically reduces the workload for the PDP.

This shows that architecting a low-latency engine isn't just about infrastructure; it starts with the design of the policy model itself.

Conclusion

In this lesson, we have architected a low-latency policy evaluation engine by combining fundamental distributed systems principles with specialized techniques from cutting-edge authorization systems.

Key Takeaways:

• Architectural Components: A policy engine is typically composed of a PEP (gatekeeper), PDP (decision-maker), and PIP (information-fetcher).
• Deployment Patterns: The choice between an embedded library, sidecar, or centralized service is a critical trade-off between latency and manageability.
• Caching is Paramount: Low latency is impossible without aggressive, multi-level caching of decisions, intermediate results, and attributes. Techniques like timestamp quantization and cache stampede protection are essential.
• Distribution for Scale: Global scalability requires replicating data and policies close to users. Clever consistency protocols, like Zanzibar's zookie mechanism, are the key to enabling low-latency reads from local replicas.
• Language Design Matters: A constrained, stateless, and loop-free policy language is itself a performance feature that enables bounded evaluation times and optimizations like policy slicing.

Preview of the Next Lesson:

We have now designed the core models (RBAC/ABAC) and the high-performance engine (PDP) for our identity platform. The next step is to enable this platform to communicate authorization decisions to external applications. In our next lesson, we will begin this process by addressing the learning outcome: "Design the SAML assertion generation process, including attribute mapping and XML digital signature creation." We will see how the policies we've defined can be used to populate the claims and attributes within a standard federated identity protocol.