Hello! Welcome to the start of your journey into the world of Conflict-free Replicated Data Types (CRDTs). Given your goal of building a sophisticated real-time, offline-first collaborative application, our first step is to build a solid foundation. This initial lesson will demystify the environment where CRDTs operate: distributed systems. We'll explore what they are, why they're central to modern web development, and how a "local-first" perspective on them is crucial for achieving your project goals.
By the end of this lesson, you'll be able to explain what a distributed system is, using examples that are directly relevant to the web applications you build every day. This understanding will be the bedrock for everything that follows, as the core challenge that CRDTs solve is that of managing data across a distributed system without a central coordinator.
From Simple to Scalable: The Web as a Distributed System
At its heart, any application involving more than one computer communicating over a network is a distributed system. Even the most basic web application you've built—a client (browser) fetching data from a server—is a two-node distributed system. The client and server are separate components, running on different machines, that coordinate their actions by passing messages (HTTP requests and responses).
As you know from your experience with performance optimization and scaling applications, this simple model rarely holds up under heavy load. To handle thousands or millions of users, we can't just use a more powerful server; we need more servers. This is where the complexity—and power—of distributed systems becomes more apparent.
Consider a typical architecture for a high-traffic web application:
Each box in this diagram represents a separate computer (or a process that could be on a separate computer). When a user interacts with the application, their request might be handled by any one of the frontend servers. The data they are trying to access could be on the primary database, while a replica of it exists on the secondary. All these independent parts must work together seamlessly to provide a coherent experience.
This setup offers two key advantages:
- Scalability: If traffic increases, you can simply add more servers to the "web frontend" or "HTTP service" pools.
- Fault Tolerance: If one web server crashes, the load balancer redirects traffic to the others. If the primary database fails, the secondary can take over.
However, this distribution also introduces significant challenges. How do you ensure the secondary database has the exact same data as the primary? What happens if one of the HTTP services fails mid-operation? The complexity of these systems led computer scientist Leslie Lamport to offer a now-famous definition.
To hear this definition and see how it applies to services you use daily, watch the following short clip from a talk by Martin Kleppmann, a leading researcher in this field.
The past, present, and future of local-first - Martin Kleppmann (Local-First Conf)
This video is "The past, present, and future of local-first" from Local-First Conf. In this segment, Martin Kleppmann introduces a classic, insightful definition of a distributed system.
Please watch from the beginning of the clip. He quotes Leslie Lamport: "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." This perfectly captures the core challenge and user experience of interacting with complex, interdependent services.
When GitHub or AWS has an outage, large parts of the internet become unusable for developers and users alike. This is Lamport's definition in action. You, as a user, have no idea which specific server in Amazon's vast infrastructure failed, but its failure prevents you from doing your work.
A New Perspective: Local-First Architectures
The architecture shown in the diagram above is "cloud-centric." The servers in the data center hold the "source of truth," and clients are primarily just for display. This model has served the web well, but as you know from your work on performance, it has drawbacks, especially for rich, interactive applications. Every significant action can require a round trip to the server, resulting in loading spinners and a dependency on a stable internet connection.
This leads to a different way of thinking about distributed systems, one that is essential for your goal of building an offline-first collaborative app: the local-first architecture. In this model, the distributed system isn't just a cluster of servers in a data center; it's a network of peers that includes the users' own devices. Each user's device holds a full copy of the relevant data and is a first-class citizen capable of creating and modifying it.
The following presentation by Alex Good, a maintainer of the Automerge CRDT library, provides an excellent perspective on this shift. It begins with a story that will likely feel very familiar: the moment a simple, local-only application needs to support multiple devices.
Local First – How to Build Software Which Still Works after the Acquihire - InfoQ
In this transcript, Alex Good explores the fragility of cloud-centric software and makes the case for a local-first approach.
Please read the section titled Collaborative Software is Fragile. Pay close attention to the user experience problems he identifies (loading spinners, loss of creative privacy) and the issue of user agency (what happens when a service is shut down). This section grounds the abstract concept of a distributed system in the tangible trade-offs you face as an application architect.
As the reading highlights, the cloud-centric model makes collaboration and offline usage feel like afterthoughts. The local-first approach, by design, treats the user's device as the primary source of truth. The network is used for synchronizing with other devices, but it's not a prerequisite for working with the application.
Martin Kleppmann formalizes this idea with a set of principles. The next video clip builds on his introduction and defines what makes an application truly local-first.
The past, present, and future of local-first - Martin Kleppmann (Local-First Conf)
We return to Martin Kleppmann's talk, where he outlines the key properties of local-first software.
Watch the segment from this clip. He explains that local-first software must be: Multi-device: It's not just local-only; it synchronizes data. Network-optional: It works seamlessly offline. "Incredible Journey-proof": The application continues to function even if the original developer's servers are shut down permanently. This last point is crucial. It implies that the application's core logic and the user's data are not dependent on a proprietary, centralized service.
The Central Challenge: Managing Distributed State
Whether your system is a massive server cluster or a peer-to-peer network of collaborative clients, the fundamental problem remains the same: how do you manage state across all the different nodes?
In a local-first system, this challenge is front and center. If two users are offline and edit the same piece of data, what happens when they reconnect? How are their changes merged? Whose change "wins"? A naive approach can quickly lead to data loss or corruption.
To make this concrete, let's explore an analogy that you're very familiar with: Git. Git is, in many ways, a local-first system. You have a complete copy of the repository on your machine, you can commit changes offline, and you can synchronize with any number of remotes.
In the next reading, Alex Good walks through a thought experiment: what if we tried to build a collaborative app using Git as the backend?
Local First – How to Build Software Which Still Works after the Acquihire - InfoQ
This reading continues from the previous one, exploring how the principles of a system like Git can be applied to application data, and where that analogy breaks down.
In the section "Building Generic Synchronization Infrastructure," read from the beginning, where he introduces Git as a local-first system, up to the end of the paragraph describing finer-grained changes. This section is key because it uses the familiar concept of merge conflicts in Git to illustrate the problem of concurrent edits in any distributed system, and it points toward the solution: moving from coarse, file-level snapshots to fine-grained, structured operations.
As the reading explains, using Git directly for real-time data synchronization is a "terrible idea" because its conflict resolution mechanism is designed for human developers merging source code, not for applications automatically merging structured data. A simple concurrent edit on a JSON file results in a merge conflict that the application can't resolve on its own.
This is precisely the problem that CRDTs are designed to solve. They are data structures that have a well-defined, automatic, and conflict-free merge function, enabling the kind of seamless, local-first collaboration that both you and your users want.
Conclusion
In this lesson, we've established a foundational understanding of distributed systems and their relevance to your goals as a frontend developer.
Here are the key takeaways:
- A distributed system consists of independent components on networked computers that coordinate to achieve a common goal. This ranges from simple client-server web apps to large-scale, fault-tolerant backend architectures.
- For building modern collaborative and offline-capable applications, the local-first architectural model is paramount. It treats user devices as primary nodes in the distributed system, rather than just thin clients for a central server.
- The core challenge in any distributed system, especially a local-first one, is managing state synchronization and concurrency. How do you ensure all nodes eventually agree on the state of the data, especially when they can all make changes independently?
You've seen that traditional tools like Git, while illustrating the principles of local-first, are not suited for automatic conflict resolution of application data. This sets the stage for our next topic. In the upcoming lesson, we will dive into the fundamental trade-offs involved in managing distributed state by exploring the concepts of strong and eventual consistency. This will lead us directly to the doorstep of CRDTs, which are the key to guaranteeing eventual consistency in a decentralized environment.