Sequential vs. Relaxed Memory Models

1. Sequential Consistency: The Intuitive Model

Let's start with memory_order_seq_cst. This is the default for all C++ atomic operations if you don't specify an order, and for good reason: it behaves exactly how you would intuitively expect a multithreaded program to work.

Sequential consistency provides two powerful guarantees:

1. There is a single, total global order of all sequentially consistent operations across all threads.
2. All threads agree on and observe this same order.

Think of it as all threads reading from and writing to a single, shared timeline. An event happens, it's appended to the end of the global log, and every other thread will see it in that exact position.

The following video section explains this concept clearly.

Let's look at a classic example to see what this means in practice.

#include <atomic>
#include <thread>
#include <cassert>

std::atomic<bool> x = {false};
std::atomic<bool> y = {false};
std::atomic<int> z = {0};

void write_x_then_y() {
    x.store(true, std::memory_order_seq_cst); // Operation 1
    y.store(true, std::memory_order_seq_cst); // Operation 2
}

void read_y_then_x() {
    while (!y.load(std::memory_order_seq_cst)); // Operation 3
    if (x.load(std::memory_order_seq_cst)) {    // Operation 4
        ++z;
    }
}

int main() {
    std::thread a(write_x_then_y);
    std::thread b(read_y_then_x);
    a.join();
    b.join();
    // In this scenario, 'z' will be 1. The 'if' will always be true.
}

Because of the global ordering guarantee of seq_cst, if Operation 3 (y.load()) sees the value true written by Operation 2 (y.store(true)), it is guaranteed that Operation 4 (x.load()) will see the value true written by Operation 1 (x.store(true)). The order of writes in write_x_then_y is preserved and observed by read_y_then_x. An outcome where y is true but x is false is impossible.

2. The Problem: Reality is More Relaxed

If sequential consistency is so intuitive, why use anything else? The answer, as always in systems programming, is performance. Modern CPUs do not, by default, provide sequential consistency because it is expensive. To hide the latency of slow main memory, each CPU core uses its own caches and store buffers.

When a core performs a write, it might just place the data in its local store buffer and continue executing other instructions. The data is only "flushed" to main memory later. This reordering is invisible and harmless in a single-threaded context, but it breaks the intuitive model in a multithreaded one.

The following article explains this beautifully.

As the article shows, consider this scenario:

Initially, x and y are false. Intuitively, it seems impossible for both r1 and r2 to end up false. At least one of the stores must happen first, right?

Wrong. With store buffers, this can happen:

1. Core 1 executes x.store(true). The value true goes into its local store buffer, but x in main memory is still false.
2. Core 2 executes y.store(true). The value true goes into its local store buffer, but y in main memory is still false.
3. Core 1 executes r1 = y.load(). It reads from main memory (since Core 2's store is still in its buffer) and gets false.
4. Core 2 executes r2 = x.load(). It reads from main memory (since Core 1's store is still in its buffer) and gets false.

The result is r1 = false and r2 = false. This is a direct consequence of the hardware's relaxed memory model. memory_order_seq_cst forbids this by forcing the CPU to issue memory fences that flush these buffers and establish a global order, which incurs a performance penalty.

3. Relaxed Ordering: Embracing the Chaos

This brings us to memory_order_relaxed. This memory order essentially tells the compiler and CPU: "I know you reorder things. For this operation, you have my full permission. Do whatever is fastest."

memory_order_relaxed provides only two, very minimal guarantees:

1. Atomicity: The operation itself is indivisible. You won't see a "half-written" value.
2. Modification Order: All threads agree on the modification order of a single atomic variable. If a thread writes 10, then 20, then 30 to x, no other thread can observe x as 30 and later observe it as 10.

It provides no ordering guarantees relative to any other memory operations, atomic or not. It does not create a happens-before relationship and cannot be used for synchronization.

Let's watch another segment of the "Beyond Sequential Consistency" talk, which perfectly illustrates the "chaos" of relaxed ordering.

The key takeaway is that with relaxed ordering, operations can be reordered so freely that the execution sequence can be completely different from the program's source code order.

void write_x_then_y_relaxed() {
    x.store(true, std::memory_order_relaxed);
    y.store(true, std::memory_order_relaxed);
}

void read_y_then_x_relaxed() {
    while (!y.load(std::memory_order_relaxed));
    if (x.load(std::memory_order_relaxed)) { // Is this guaranteed to be true?
        ++z;
    }
}

4. The Performance Payoff and Hardware Reality

Why would anyone use relaxed if it's so chaotic? Because it's fast. And why is seq_cst slow? Because it emits stronger instructions.

Let's connect this to the low-level details you're interested in. The C++ memory model is an abstraction layer on top of hardware. The performance cost comes from the real machine instructions the compiler must generate to satisfy the ordering guarantees.

These barriers are expensive. They force the CPU pipeline to stall and synchronize with other cores. By using relaxed or acquire-release, you are telling the compiler it can avoid these expensive barriers, giving you a significant performance boost. The following talk graphically demonstrates this cost.

Summary: A Spectrum of Guarantees

You can now see the full spectrum of C++ memory orders. They are not arbitrary rules but a carefully designed system to give you control over the trade-off between correctness and performance.

Here is a summary table contrasting the two models we focused on today:

For your telemetry engine's SPSC queue, we chose acquire-release because we don't need a global order. We only need the producer to synchronize with the consumer. Using seq_cst would be correct, but it would be unnecessary overhead. This is precisely the kind of informed, performance-oriented decision you need to make in systems programming.

Conclusion

You now have a solid grasp of the two extremes of the C++ memory model.

Key Takeaways:

• Sequential Consistency (seq_cst) is the most intuitive and strictest model. It guarantees a single global order of operations visible to all threads, but this comes at a significant performance cost due to the need for strong memory fences.
• Relaxed Ordering (relaxed) is the fastest and loosest model. It only guarantees atomicity and modification order for a single variable, allowing the compiler and CPU to reorder operations freely for maximum performance. It provides no synchronization.
• The difference stems from hardware optimizations like store buffers, which break the intuitive sequential model. Memory orders give you control over how and when the CPU synchronizes its state with other cores.

What's Next?

We've covered reading and writing to atomics with different memory orders. In our next lesson, we will explore a new type of atomic operation: the read-modify-write, specifically compare-and-swap. You'll use this powerful primitive to implement a spin-lock, applying your knowledge of memory ordering to ensure its correctness and performance.