Diagnosing Replication Lag Consistency Anomalies

Good to see you again. Last time, you compared the ways a leader can represent and send changes to followers: SQL statements, physical write-ahead-log records, logical row changes, and trigger-captured changes. Regardless of that format, an asynchronously updated follower can temporarily be behind the leader.

This lesson focuses on what that delay looks like to users. A replicated system may have committed the correct write and still give a user an apparently incorrect answer because their read was routed to a replica that has not applied that write yet. You will learn to distinguish three guarantees that address different versions of this problem:

• read-after-write: I should see my own confirmed update;
• monotonic reads: my view should not move backward in time;
• consistent-prefix reads: effects should not appear before their causes.

---

Replication lag creates several different “stale read” problems

In single-leader replication, writes go to the leader. With asynchronous replication, the leader can acknowledge a write before every follower has received and applied it. At any moment, followers may therefore represent different points in the leader’s history.

Let $L$ be the leader’s latest committed replication position, and let $p_i$ be the latest position applied by follower $i$. During lag, some follower has:

$$p_i<L$$

A load balancer that sends each read to any available follower may send two otherwise related requests to replicas at different positions. That routing choice is enough to create surprising behavior, even if every server is functioning normally.

The key diagnostic question is not merely “Was the read stale?” It is:

What relationship between operations did the application need to preserve?

The three guarantees below answer that question at increasing levels of scope.

Guarantee	Relationship protected	User-visible failure
Read-after-write	A session’s write, then its later read	“I saved it, but it is gone.”
Monotonic reads	An earlier read, then a later read in one session	“I saw it, then it disappeared.”
Consistent prefix	Causally ordered writes, as observed by any reader	“I saw the reply before the message it replies to.”

These are session guarantees, not full strong consistency. They constrain particular anomalies while still allowing replicas to serve some stale reads.

---

Read-after-write: a writer must see their own committed work

Read-after-write consistency, also called read-your-writes, means that once a user receives confirmation that their write succeeded, later reads in that user’s session must include that write.

Suppose Maya submits a comment:

1. Maya’s application sends the comment to the leader.
2. The leader commits it and returns success.
3. The application refreshes the comment thread.
4. The refresh is routed to a follower that has not received Maya’s comment.

The follower truthfully reports no such comment in its current local state. But the overall system has violated the experience Maya expects: success was acknowledged, yet her own change is missing.

Formally, if Maya’s acknowledged write occupies replication position ${\ell }_w$, every subsequent protected read must go to a server whose applied position is at least ${\ell }_w$:

$$p_i\ge {\ell }_w$$

This does not mean Maya must see every other user’s latest activity. Another person’s recent comment may still be absent. The guarantee is narrowly about preserving the visibility of Maya’s own prior writes.

Common ways to provide it

Read specific user-modifiable data from the leader.
For example, route reads of a user’s profile, settings, drafts, or submitted posts to the leader when that same user is the only person authorized to change them. The leader necessarily includes the successful write.

This is clear and dependable, but it does not scale well if almost every user-facing read becomes a leader read.

Temporarily route that user’s reads to the leader after a write.
The application records that the user wrote recently and uses the leader for a short window. This can reduce leader load compared with permanently pinning all personal reads there. But a fixed duration is only a heuristic: lag may occasionally last longer than the selected window.

Carry a required replication position with the session.
After the write, the application retains a version or log position. A read can use a follower only after that follower has caught up to the required position. This avoids guessing based on elapsed time. Systems use different names and mechanisms for these positions, but the underlying idea is the same: select a replica that has applied at least the write the session depends on.

There are two operational complications worth noticing:

• Cross-device sessions: if Maya posts on her phone and reads on her laptop, the application must propagate the “required position” or equivalent session information between devices.
• Geographic replicas: a nearby follower may be fast for reads but still behind a distant leader. Routing a protected read to the leader can increase latency; routing it locally can violate read-after-write unless the local replica has caught up.

The Cornell paper gives a useful precise framing: a protected read must use a database copy that includes all earlier writes from that same session.

---

Monotonic reads: a reader must not go backward

A user can encounter a different anomaly without writing anything. Imagine that Noor opens a discussion thread.

1. The first request reaches follower A, which has already received a new comment.
2. Noor sees the comment.
3. A later page refresh reaches follower B, which is further behind.
4. The comment is absent.

From Noor’s perspective, the system has gone backward. The comment appears to have been deleted, although nothing was deleted.

Monotonic reads require that a user’s later reads never observe a state older than the state that the user has already observed. A user may initially receive an old value, but the system must not later give that same user an even older view.

This is different from read-after-write:

• Read-after-write begins with the user’s write.
• Monotonic reads begin with the user’s earlier read.

Consider a public news feed. A reader may not have posted any content, so read-after-write says nothing about their experience. Monotonic reads still matters: if they have already seen an article or comment, a refresh should not make it vanish just because a different, less current replica answered.

A practical mitigation: sticky replica assignment

A common approach is to assign each user to a replica for the duration of a session, or more persistently using a hash of the user ID. If Noor always reads from follower A, and follower A applies changes in order, Noor’s observations naturally move forward as A catches up.

This is often called session stickiness. It gives different users different replicas, preserving read scaling while preventing random replica changes from making one user’s view regress.

However, stickiness is not a complete solution:

• The assigned replica can fail or become unavailable.
• A user may switch devices or regions.
• A replica may be too far behind to serve the user’s needs.
• A failover to a different follower must avoid moving the user to an older state.

A stronger variant carries the latest version the session has seen. The router may use any replica, but only one known to be at least that current. In other words, it turns “stay on one replica” into “never select a replica behind my observed state.”

---

Consistent prefix: effects must not precede causes

The third anomaly is about causality between writes, rather than the history of one user’s session.

Suppose a chat contains these two messages:

1. Arun: “Can you review the deployment plan?”
2. Bea: “Yes, I will review it this afternoon.”

Bea’s reply is causally dependent on Arun’s question. A reader who sees Bea’s reply but not Arun’s question sees an incoherent conversation. The reader may reasonably infer that Bea has somehow responded before the question existed.

A consistent-prefix read guarantees that if writes were causally ordered, readers observe them in that same order. If a read includes the second event, it must also include the first.

For causally related writes $W_1$ and $W_2$, where $W_1$ happened before and influenced $W_2$, an observed state must not contain $W_2$ while omitting $W_1$.

The word prefix matters. Imagine a history ordered as:

$$W_1,W_2,W_3,\dots$$

A replica may be behind and show only $W_1$, or $W_1$ and $W_2$. Those are valid prefixes. Showing $W_2$ without $W_1$ is not.

Why partitioning makes this harder

Within a single leader’s ordered replication log, a follower normally applies changes in that order. If it has applied the second log entry, it has already applied the first. Thus a single replicated log naturally has a prefix structure.

The challenge appears when related writes land on different partitions:

• Arun’s question is written to partition A.
• Bea reads the question and writes her reply to partition B.
• Partition B’s follower is caught up, while partition A’s follower lags.
• An observer reads both partitions and sees the reply before the question.

Each partition may be correctly replicating its own data. The error is that there is no single global order ensuring that the second partition waits for the first partition’s causally relevant update before exposing the reply.

This is why consistent prefix is especially important for conversations, threaded comments, timelines, audit trails, and dependent workflow events.

Ways to reduce or prevent it

Keep causally related data together when feasible.
If a conversation and its replies are assigned to the same partition, they can use one ordered log. This is often an effective data-modeling choice, though it can conflict with other needs such as load balancing.

Track causal dependencies explicitly.
A system can record that Bea’s reply depends on Arun’s question. A replica or reader then delays exposing the reply until the prerequisite is available. This idea leads toward causal consistency, which will be developed later in the course.

Read a coordinated snapshot.
For some queries, the system can ensure that the multiple partitions being combined correspond to a causally coherent point. This is more demanding than simply sending independent requests to whichever replica is nearby.

Consistent-prefix reads do not require every unrelated write in the entire system to have one universal order. The requirement is specifically that observable causes precede their effects.

---

A diagnostic method: identify the violated dependency

When someone reports “the database was inconsistent,” translate the report into an operation history before selecting a fix.

Symptom	Operation pattern to look for	Likely diagnosis	Appropriate direction
“I saved my profile, refreshed, and the old profile returned.”	Same session writes, then reads its own data	Read-after-write violation	Leader routing, session version, or caught-up replica
“I saw a notification, refreshed, and it disappeared.”	Same session reads a newer replica, then an older one	Monotonic-read violation	Sticky routing or a minimum read version
“The reply appears before the original post.”	One write depends on another, often across partitions	Consistent-prefix violation	Co-locate data, preserve dependencies, or use a coordinated read

There is a useful distinction in the first two rows:

• A read-after-write failure breaks a promise between my write and my later read.
• A monotonic-read failure breaks a promise between my earlier read and my later read.

A system can satisfy one and fail the other. For example, routing a user to the leader right after they submit a post may give them read-after-write consistency. If later requests are randomly routed among followers without tracking what they have seen, monotonic reads can still fail.

Similarly, sticky routing can help monotonic reads even for a reader who never writes. It does not by itself guarantee read-after-write across devices, because the second device may not use the same sticky replica or know the first device’s required version.

Finally, none of these guarantees means every read sees the very latest committed data globally. They are targeted promises that make eventually consistent replication usable for human-facing interactions.

---

Key takeaways

Replication lag becomes a user-facing correctness issue when requests move among replicas at different replication positions.

• Read-after-write consistency ensures that a user’s later reads include that user’s acknowledged earlier writes. It prevents the “I saved it, but it disappeared” experience.
• Monotonic reads ensure that later reads in one session do not observe a state older than a previously observed state. They prevent the system from appearing to move backward in time.
• Consistent-prefix reads preserve causal order: readers cannot see an effect without its cause. This is particularly challenging when related writes are stored and replicated independently across partitions.
• Leader routing, replica stickiness, session-carried versions, causal-dependency tracking, and careful partitioning are tools for providing the appropriate guarantee. The right choice depends on the relationship the application needs to preserve.

Next, you will move from lag-induced read anomalies to multi-leader replication, where several nodes can accept writes and the central challenge becomes resolving conflicting concurrent updates.