Hello! Welcome back to our course on preparing for senior backend engineer interviews.
In our last lesson, we dug deep into database indexing, comparing the trade-offs between B-Trees (optimized for reads) and LSM-Trees (optimized for writes) on a single database node. We established how to make one machine as efficient as possible.
Today, we address the next logical question: What happens when a single machine is no longer sufficient? Whether it's to handle more read traffic, ensure the system stays online if a server fails, or reduce latency for a global user base, we need to move beyond a single node. This is where database replication comes in.
This lesson directly addresses the learning outcome: Describe database replication patterns (master-slave, master-master) and their trade-offs. Understanding these patterns is fundamental to designing scalable, available, and fault-tolerant systems—all critical topics in a senior-level system design interview.
1. The "Why": Goals of Database Replication
Before we dive into how replication works, let's establish why we need it. Storing your data on a single machine creates a single point of failure and a performance bottleneck. Replication involves keeping copies of your data on multiple machines to solve these problems.
🔴 How DataBase Replication works? WATCH THIS before your System Design Interview
To start, let's watch an overview from the channel 'Rachit' that explains the core motivations for distributing data across multiple machines. It differentiates replication from partitioning (which we'll cover in a future lesson) and sets the stage for our discussion.
Please watch from the beginning to 07:03. Focus on the three primary reasons for using multiple machines: scalability, availability, and latency.
As the video highlighted, the main goals of replication are:
- High Availability / Fault Tolerance: If one database server (a replica) goes down, the system can continue operating by using other replicas. This prevents a single hardware failure from taking down your entire application.
- Scalability (for reads): Read requests can be distributed across multiple replicas. This allows the system to handle a much higher volume of read traffic than a single server could.
- Lower Latency: For applications with a global user base, replicas can be placed in different geographical regions. Users are then served by the nearest replica, which significantly reduces network latency and improves user experience.
Now, let's look at the common architectural patterns for implementing replication.
2. Single-Leader Replication (Master-Slave)
This is the most common replication model, used by many popular databases like PostgreSQL, MySQL, and SQL Server. The architecture is straightforward: one node is designated as the leader (or master), and all other nodes are followers (or slaves).
The rules are simple:
- All write operations (INSERT, UPDATE, DELETE) must go to the leader.
- The leader logs these changes and sends them to its followers.
- Followers apply the changes from the leader in the same order to keep their data copies in sync.
- Read operations can be served by the leader or any of the followers.
This model is excellent for read-heavy workloads because you can scale out reads by simply adding more followers.
Key Trade-off: Synchronous vs. Asynchronous Replication
A critical decision in this architecture is how the leader propagates changes. This choice directly impacts your system's durability and performance.
🔴 How DataBase Replication works? WATCH THIS before your System Design Interview
Let's continue with the video from 'Rachit' to explore this fundamental trade-off. It also begins to touch on the complexities of failure handling.
Please watch from 08:08 to 12:24. Pay close attention to the pros and cons of synchronous vs. asynchronous replication. This is a classic interview discussion point.
Let's summarize the trade-offs discussed:
| Replication Mode | Pros | Cons |
|---|---|---|
| Synchronous | - High Durability: A write is only confirmed once it's saved to the leader AND one or more followers. No data is lost if the leader fails. | - Lower Write Performance: The leader must wait for confirmation from a follower, increasing write latency. - Reduced Availability: If the synchronous follower is down or slow, the leader cannot process writes. |
| Asynchronous | - High Write Performance: The leader confirms the write immediately after processing it locally and sends updates to followers in the background. | - Potential for Data Loss: If the leader fails before replicating recent writes to any follower, those writes are lost. - Replication Lag: Followers may not have the most up-to-date data, leading to clients reading stale data. |
Most systems use asynchronous or a hybrid semi-synchronous approach (where the leader waits for just one follower to confirm) to balance performance and durability.
Handling Failures in Master-Slave Setups
Discussing failure scenarios is where you can demonstrate senior-level thinking. The most critical failure is when the leader goes down. This process is called failover.
🔴 How DataBase Replication works? WATCH THIS before your System Design Interview
Now, let's look at the operational complexities of handling failures. This is a core part of managing a replicated database system.
Watch the segment from 12:24 to 18:51. Focus on the steps for leader failover and the potential problems that can arise, such as data loss in asynchronous mode and the 'split-brain' problem.
Key points on failover:
- Detection: A system (or a human) must first detect that the leader has failed, often using a heartbeat mechanism.
- New Leader Election: A new leader must be chosen from the followers. The best candidate is the follower with the most up-to-date data to minimize data loss.
- Reconfiguration: The system must be reconfigured. The application needs to send writes to the new leader, and other followers need to start following the new leader.
This process is complex and fraught with potential issues, especially in asynchronous setups, including data loss and the dangerous "split-brain" scenario where two nodes believe they are the leader.
To round out your understanding, it's useful to have a concise summary of this model.
Master-Slave vs Master-Master: Key Differences
The article 'Master-Slave vs Master-Master' from Hokstad Consulting provides a great summary of the pros, cons, and ideal use cases for this model.
Please read the section 'Master-Slave Replication: How It Works', including the subsections on its architecture, pros and cons, and when to use it.
3. Multi-Leader Replication (Master-Master)
What if a single leader for writes becomes a bottleneck, or if you need to accept writes in multiple geographic locations simultaneously? This is where multi-leader replication comes in.
In this model, you have two or more nodes that can accept writes. Each leader acts as a follower to the other leaders.
The Core Challenge: Conflict Resolution
The biggest advantage of multi-leader replication—accepting writes in multiple places—is also its biggest challenge. What happens if two users update the same record at the same time on two different leaders? This is a write conflict, and you need a strategy to resolve it.
Google SWE teaches systems design | EP3: Multileader replication
This video from 'Jordan has no life' gives an excellent, in-depth look at multi-leader replication, with a strong focus on the all-important topic of conflict resolution.
Watch from 00:35 to 04:35. This segment introduces the concept, discusses topologies, and covers the two simplest conflict resolution strategies: conflict avoidance and 'Last Write Wins' (LWW). Pay attention to the significant drawbacks of LWW.
As the video explains, Last Write Wins (LWW) is a common but problematic strategy. It's simple to implement but can lead to silent data loss, as an earlier write can be completely overwritten without the user ever knowing.
A more robust way to handle conflicts is to detect them and then merge them, either by the database or by the application. To do this, we need a way to determine if two writes are concurrent.
Google SWE teaches systems design | EP3: Multileader replication
Let's continue with the same video to understand a more advanced mechanism for detecting concurrent writes: version vectors.
Now watch from 04:35 to 13:08. This is a deep dive into conflict resolution and using version vectors to detect concurrent writes. Understanding this concept at a high level is a strong signal of seniority.
Version vectors are a powerful mechanism that allows a database to definitively determine if one write happened "before" another, or if they happened concurrently (i.e., without knowledge of each other). When a conflict is detected, the database can store both versions and have application logic merge them later.
For a final summary of the master-master model, let's turn back to the article.
Master-Slave vs Master-Master: Key Differences
The Hokstad Consulting article also provides a concise summary of the multi-leader model's architecture and trade-offs.
Read the section 'Master-Master Replication: How It Works', including its architecture, pros and cons, and use cases.
4. Comparison and Choosing the Right Model
In a system design interview, you will be expected to weigh these options and justify your choice based on the problem's requirements. Here's a summary of the trade-offs:
| Criteria | Master-Slave (Single-Leader) | Master-Master (Multi-Leader) |
|---|---|---|
| Write Scalability | Limited. All writes go to one master, creating a bottleneck. | High. Writes can be distributed across multiple leaders. |
| Read Scalability | Excellent. Can add many read replicas. | Good. All nodes can serve reads, but they also handle writes. |
| Consistency | Strong consistency is easier to achieve (especially with synchronous replication). Writes are ordered by a single leader. | Eventual consistency. Conflicting writes are possible, and data becomes consistent only after conflicts are resolved. |
| Complexity | Simpler to set up and manage. No write conflicts to resolve. | Much more complex. Requires robust conflict resolution logic. |
| Fault Tolerance | Master failure requires a failover process, which can cause downtime and potential data loss. | High write availability. If one leader fails, others can still accept writes. Less susceptible to downtime. |
| Use Cases | Read-heavy applications (e.g., content websites, e-commerce catalogs), systems requiring strong consistency. | Globally distributed applications needing low write latency (e.g., collaborative editing tools like Google Docs), systems requiring high write availability. |
Master-Slave vs Master-Master: Key Differences
To solidify this comparison, please review the direct comparison section in the Hokstad Consulting article. It provides an excellent table and discussion of the practical differences.
Read the sections 'Master-Slave vs Master-Master: Direct Comparison' and 'Choosing the Right Replication Model'. Focus on the criteria for making a decision.
Conclusion
You now have a framework for discussing two of the most important database replication patterns. The choice between them is a classic system design trade-off between consistency, availability, and complexity.
Key Takeaways:
- Replication is essential for building scalable and reliable systems, primarily by providing read scalability and high availability.
- Master-Slave (Single-Leader) is the most common model. It is simple, provides strong consistency, and is ideal for read-heavy workloads. Its main drawbacks are a single point of failure for writes and a write performance bottleneck.
- Master-Master (Multi-Leader) provides high write availability and is well-suited for geographically distributed systems. Its primary drawback is the significant complexity of conflict resolution, which leads to an eventually consistent model.
- The choice is not about which is "better," but which is the right fit for the specific application's requirements regarding consistency, latency, and availability.
In our next lesson, we will discuss another crucial technique for scaling databases: sharding. While replication is about copying data, sharding is about partitioning it. We'll learn how to split massive datasets across multiple servers, a technique often used in conjunction with replication.