Understanding Memory Orderings

The Spectrum of Guarantees

Think of C++ memory orders as a spectrum. On one end, you have maximum safety and ease of reasoning; on the other, you have maximum performance and compiler freedom. Your job as a systems programmer is to pick the right point on this spectrum for the task at hand.

We will not cover memory_order_consume, as it's notoriously difficult to use correctly and is currently treated as memory_order_acquire by all major compilers.

Choosing the Right Tool for the Job: Common Scenarios

Let's walk through the most common scenarios you'll encounter and determine the right memory order for each.

Scenario 1: Independent Operations (e.g., Counters)

The Situation: You need to perform an atomic operation, like incrementing a counter, but this operation does not synchronize any other data. The only thing that matters is that the operation itself is not torn. This is common for collecting statistics, like counting events or work items.

The Choice: memory_order_relaxed

The Rationale: Since no other memory access depends on the order of this atomic operation, you don't need any synchronization barriers. relaxed provides the bare minimum—atomicity—at the lowest possible performance cost. It tells the compiler and CPU, "Just make this single operation atomic; I don't care what you do with the surrounding code."

Here's what that looks like in code, from the "Multithreading in Modern C++" article:

// from https://dev.to/cear/multithreading-in-modern-c-lock-free-programming-memory-ordering-and-atomics-4cek
std::atomic<int> counter{0};

void incrementCounter() {
    for (int i = 0; i < 100000; ++i) {
        // We only care that the counter is correctly incremented.
        // No other data is being synchronized by this operation.
        counter.fetch_add(1, std::memory_order_relaxed);
    }
}

Scenario 2: Producer-Consumer Synchronization

The Situation: This is the most common synchronization pattern. One thread (the "producer") prepares some data and then needs to signal to another thread (the "consumer") that the data is ready. This is the core logic you will use in your project's SPSC (Single-Producer, Single-Consumer) ring buffer.

The Choice: memory_order_release for the producer's signal, and memory_order_acquire for the consumer's check.

The Rationale: This pair creates a "happens-before" relationship.

• store(..., memory_order_release): Guarantees that all memory writes that happened before this store in the producer thread are visible to the consumer. It acts like a barrier preventing prior writes from being reordered past it. You are "releasing" your changes to the world.
• load(..., memory_order_acquire): Guarantees that all memory reads that happen after this load in the consumer thread will see the data published by the producer. It acts like a barrier preventing subsequent reads from being reordered before it. You are "acquiring" the changes from the producer.

The producer-consumer pattern is clearly illustrated in this code example.

Scenario 3: Read-Modify-Write Operations as Fences

The Situation: You need to perform a read-modify-write (RMW) operation, like fetch_sub or compare_exchange, that needs to both acquire changes from other threads and release its own. A classic example is decrementing a reference count. When you decrement, you release your own modifications to the shared object, but if your decrement brings the count to zero, you must acquire all other threads' modifications before you can safely delete the object.

The Choice: memory_order_acq_rel

The Rationale: This ordering combines the properties of acquire and release into a single, powerful operation. It creates a full memory fence, preventing any memory reordering in either direction across the atomic operation.

Scenario 4: Maximum Safety and Complex Algorithms

The Situation: You are implementing a complex algorithm with multiple interacting atomic variables, and reasoning about the partial ordering of acquire-release is too difficult or error-prone. Or, you're just starting out and want to prioritize correctness over performance.

The Choice: memory_order_seq_cst

The Rationale: seq_cst is the default for a reason: it's the most intuitive. It establishes a single global order of all sequentially consistent operations that all threads agree on. This eliminates many subtle reordering bugs that can occur with weaker memory models, making your code easier to reason about. However, this safety comes at a steep performance cost.

The rule of thumb is: start with seq_cst if you are unsure. If profiling later reveals the atomic operation to be a bottleneck, you can then carefully analyze the synchronization requirements and relax the ordering to acquire/release or even relaxed if it is safe to do so.

A Decision-Making Heuristic

You can use the following questions to guide your choice:

1. Is this atomic operation synchronizing other memory?

   • No: It's likely just a counter or a simple flag. Use memory_order_relaxed.
   • Yes: Proceed to the next question.

2. Is this a "one-way" synchronization, where a producer is publishing data to a consumer?

   • Yes: Use memory_order_release on the producer's store and memory_order_acquire on the consumer's load. This is your future SPSC queue.
   • No: Proceed to the next question.

3. Is this a read-modify-write operation that needs to both read others' changes and publish its own?

   • Yes: This is a two-way synchronization point. Use memory_order_acq_rel.
   • No: Proceed to the next question.

4. Is the algorithm complex, involving multiple atomics, or are you prioritizing safety and ease of reasoning over absolute performance?

   • Yes: Default to memory_order_seq_cst. Write correct code first, then optimize if necessary.

template<typename T, size_t Size>
class SPSCQueue {
private:
    std::array<T, Size> buffer_;
    std::atomic<size_t> writePos_{0};
    std::atomic<size_t> readPos_{0};

public:
    bool push(const T& value) {
        size_t currentWrite = writePos_.load( /* memory order here */ );
        size_t nextWrite = (currentWrite + 1) % Size;

        // Check if the queue is full by reading the consumer's position.
        if (nextWrite == readPos_.load( /* memory order here */ )) {
            return false; // Queue is full
        }

        buffer_[currentWrite] = value; // Write the data

        // Publish the new write position.
        writePos_.store(nextWrite, /* memory order here */ );
        return true;
    }
    // ... pop method ...
};

bool push(const T& value) {
    // 1. We can use relaxed for this load. The `writePos_` is only ever modified
    //    by this thread (the single producer), so we don't need to synchronize
    //    with any other thread to read its current value.
    size_t currentWrite = writePos_.load(std::memory_order_relaxed);
    size_t nextWrite = (currentWrite + 1) % Size;

    // 2. We must use acquire when reading the consumer's position (`readPos_`).
    //    This is because the consumer thread updates `readPos_` with a release
    //    operation. Using acquire ensures we get the latest value and prevents
    //    reordering of this load with subsequent operations.
    if (nextWrite == readPos_.load(std::memory_order_acquire)) {
        return false; // Queue is full
    }

    buffer_[currentWrite] = value; // Write the data

    // 3. We must use release when publishing the new write position. This creates
    //    a happens-before relationship. It guarantees that the write to
    //    `buffer_[currentWrite]` is visible to the consumer *before* the consumer
    //    sees the updated `writePos_`.
    writePos_.store(nextWrite, std::memory_order_release);
    return true;
}

Conclusion

You've now established a robust mental model for choosing memory orders. This isn't just an academic exercise; it's a fundamental skill for building the high-frequency telemetry system you've set out to create. Every atomic operation in your lock-free ring buffer will require a deliberate choice of memory ordering to ensure both correctness and zero-overhead performance.

Key Takeaways:

• relaxed: For non-synchronizing atomic operations like simple counters. Highest performance, lowest safety guarantees.
• acquire/release: The workhorse for producer-consumer patterns. Use release to publish data and acquire to consume it. This is the heart of your SPSC queue.
• acq_rel: For read-modify-write operations that act as a two-way synchronization point.
• seq_cst: The safest default. Use it when correctness is hard to reason about or when performance is not critical. It's better to be correct and slow than fast and wrong.
• Choosing a memory order is a form of communication. It tells other programmers (and your future self) about your intentions for synchronization.

What's Next?

You now understand when to use the different memory orderings. In the next lesson, we will dig deeper into the why. We will formally distinguish between sequential consistency and relaxed memory models, exploring the specific kinds of instruction reordering that the hardware and compiler are allowed to perform under a relaxed model. This will give you a much more profound appreciation for the guarantees you are giving up in the pursuit of performance.