Hello again! In our last lesson, we examined the nuances of distributed locking and saw a practical trade-off emerge: the high consistency and safety of a ZooKeeper-based lock come at the cost of higher latency compared to a faster, but less safe, Redis-based lock. This tension between performance and correctness is a recurring theme in distributed systems.
Today, we'll formalize this trade-off using the PACELC theorem. You've likely encountered the CAP theorem, but PACELC provides a more complete model by considering not just what happens during a network failure, but also the critical choices you must make during normal operation. Your goal is to learn how to articulate the PACELC theorem and use it as a powerful framework to reason about system design choicesāa key skill for discussing architectures at a senior level.
1. Beyond CAP: The Missing Trade-off
The CAP theorem, which states that a system must choose between Consistency and Availability during a Partition, is fundamental. However, its scope is limited to failure scenarios. Network partitions are relatively rare; most of a system's life is spent in a "normal" state. What guides our design choices then?
This is the question that led to the development of the PACELC theorem. As you'll see in the original paper by its creator, Daniel J. Abadi, ignoring the trade-offs during normal operation is a major oversight.
Consistency Tradeoffs in Modern Distributed Database ...
This is the seminal paper that introduced the PACELC theorem. Reading the introduction and the critique of CAP's limitations will provide the foundational "why" behind PACELC.
Please read the introduction and the section "CAP IS FOR FAILURES". Focus on these key arguments: The latency trade-off has arguably been more influential on modern database design than the CAP trade-offs. The CAP theorem imposes no restrictions during normal operation, yet systems like DynamoDB and Cassandra (which you have experience with) still choose to relax consistency. The reason isn't CAP, but something else.
Abadi's core point is that in any replicated system, a fundamental trade-off between consistency and latency exists all the time, regardless of partitions. PACELC captures this reality.
2. Deconstructing the PACELC Theorem
The PACELC theorem is best understood as a conditional statement that extends CAP. It is often phrased as follows:
If there is a Partition, a system must choose between Availability and Consistency. Else (when the system is running normally), it must choose between Latency and Consistency.
Let's break this down.
System Design Mock Interview: PACELC theorem | by Yash - Medium
This article frames the explanation in a very practical, interview-oriented way which aligns with your goals.
Read the section titled "The Setup". It provides a clear, concise definition of the theorem and its two distinct parts.
Now, let's visualize this framework before diving into each component.
The P-A-C Part: The CAP Theorem Revisited
The first half of PACELC is simply a restatement of the CAP theorem. When nodes in a distributed system cannot communicate (a partition), you must make a choice:
- PA (Prioritize Availability): The system continues to operate and respond to requests, even if it means serving potentially stale data. Systems like Amazon's DynamoDB and Apache Cassandra are classic examples. They choose to remain online for both reads and writes, accepting that data will eventually be reconciled.
- PC (Prioritize Consistency): The system prioritizes data integrity. If a node cannot confirm a write with the required number of other nodes, it will return an error or block, effectively choosing to be unavailable rather than risk inconsistency. Traditional relational databases like PostgreSQL in a clustered setup and systems built on Raft/Paxos often fall into this category.
The E-L-C Part: The Everyday Trade-off
This is the crucial extension that PACELC provides. In the absence of partitions, when a system replicates data to provide fault tolerance, it faces an inherent choice between latency and consistency.
-
EL (Prioritize Latency): To minimize response time, the system can respond to a write request after persisting it to only one or a few nodes, and then replicate the data to other nodes asynchronously in the background. This leads to very low latency for the client but introduces a replication lag, during which different nodes may have different versions of the data. This is a form of eventual consistency.
-
EC (Prioritize Consistency): To ensure strong consistency, the system will use synchronous replication. It will wait to confirm a write until that write has been successfully replicated to a quorum (or all) of the replicas. This guarantees that any subsequent read will see the latest data, but it comes at the cost of higher latency, as the client must wait for the cross-node communication to complete.
Let's make this concrete. The following chart shows benchmark results for Cassandra, a database you're familiar with. It measures the latency for write and read operations at two different consistency levels: ONE (an EL choice) and ALL (an EC choice).
Your experience with quorum writes (W + R > N) directly relates to this. Tuning these values is how you navigate the spectrum between L and C in the "Else" part of PACELC.
3. Applying PACELC to Real-World Systems
The power of PACELC lies in using it as a classification tool to understand the design philosophy of a system and to choose the right tool for a job. A system's PACELC profile is a shorthand for its core behavior.
System Design Mock Interview: PACELC theorem | by Yash - Medium
This part of the article provides excellent, practical scenarios that map directly to real-world applications.
Please read the sections "PACELCās Four Major Trade-offs" and the first two practical examples. Pay close attention to how the choice of a PA/EL system for product reviews versus a PC/EC system for bank balances is driven entirely by business requirements.
As you just read, different use cases demand different PACELC profiles:
- PA/EL Systems: These prioritize uptime and speed. Think of social media feeds, product review sections, or the shopping cart on an e-commerce site like Amazon. For these applications, being fast and always available is more important than every user seeing the absolute latest data instantly. DynamoDB and Cassandra are archetypal PA/EL systems.
- PC/EC Systems: These prioritize correctness above all else. Financial systems, inventory management, and booking systems require this profile. A transaction must be correct and globally consistent, even if it takes a few hundred milliseconds longer. Traditional RDBMS like PostgreSQL and NewSQL databases like Google Spanner or CockroachDB are PC/EC systems.
- PA/EC Systems: This is a less common but interesting hybrid. These systems prioritize availability during a partition but aim for strong consistency during normal operation. As the article notes, Google Docs is a good example: you can work offline (PA), but when online, it uses complex protocols to ensure everyone sees a consistent view (EC). MongoDB can also be configured in this way.
- PC/EL Systems: This is another hybrid where the system prioritizes consistency during a partition but favors latency in normal operation. Yahoo's PNUTS database was a well-known example.
This framework is invaluable in a system design interview. When presented with a problem, your first step is to clarify requirements. The non-functional requirements for availability, latency, and consistency directly map to a PACELC profile, which then points you toward the right class of database or technology.
4. Articulating PACELC for Interviews
Being able to define the theorem is good; being able to use it to drive a design discussion and explain it simply is what distinguishes a senior candidate.
When asked to design a system, you can use PACELC as your guide:
- Clarify Requirements: Ask questions like, "What happens if we lose connectivity between data centers? Do we fail requests or serve possibly stale data?" (This defines P-A-C). "During normal operation, how critical is it for a read to return the absolute latest write? What is our latency budget?" (This defines E-L-C).
- State Your Choice: Based on the answers, you can state, "Given these requirements, we need a PA/EL system. This leads me to consider technologies like Cassandra or DynamoDB because..."
- Explain the Trade-offs: Articulate why you made that choice. "We are accepting eventual consistency to achieve lower latency and higher availability, which is acceptable for a use case like a user's comment history."
For explaining the concept itself, having a simple analogy demonstrates clear thinking.
System Design Mock Interview: PACELC theorem | by Yash - Medium
This final section provides a simple but effective analogy for explaining PACELC.
Read the section "How Iād Explain This to a Junior Engineer". The shared spreadsheet analogy is an excellent tool to have in your pocket for an interview.
Conclusion
In this lesson, we've moved beyond the limited scope of the CAP theorem to embrace the more comprehensive PACELC framework. This model provides a lens through which we can analyze and classify nearly any distributed data store.
Key Takeaways:
- The CAP theorem only describes the trade-off between availability and consistency during a network partition.
- The PACELC theorem extends this by adding a second, crucial trade-off: the choice between latency and consistency that exists during normal operation in any replicated system.
- A system's design philosophy can be categorized by its PACELC profile (e.g., PA/EL, PC/EC), which is dictated by business requirements.
- Your practical experience with DynamoDB, Redis, and PostgreSQL directly maps to different points on the PACELC spectrum, and now you have the theoretical framework to articulate the trade-offs involved.
In our next lesson, we will dive into a specific mechanism that allows PA/EL systems like DynamoDB and Cassandra to scale horizontally: consistent hashing with virtual nodes. Understanding PACELC helps you choose what kind of system you need, and understanding consistent hashing will help you see how that system achieves massive scale.