Benchmarking SPSC Queues with `std::thread` and `std::chrono`

Part 1: Precision Timing with std::chrono

To measure performance, we need a reliable stopwatch. In C++, the standard tool for this is the <chrono> library. It provides a type-safe way to handle time, which prevents many common errors.

The library is built on three key concepts:

1. duration: A span of time, like "5 milliseconds" or "100 nanoseconds".
2. time_point: A specific moment in time.
3. clock: An object that provides the current time_point.

The most critical choice for benchmarking is which clock to use. The video below explains the fundamental components of std::chrono and highlights the crucial difference between system_clock and steady_clock.

As the video explains, system_clock tracks wall-clock time and can be adjusted by the user or operating system, even moving backward. This makes it unsuitable for measuring performance. In contrast, steady_clock is monotonic, meaning it is guaranteed to only move forward. This makes it the perfect tool for measuring elapsed time in a benchmark.

Once we have two time_points from steady_clock (a start and an end time), their difference is a duration. To interpret this duration, we often need to cast it to a specific unit, like nanoseconds or milliseconds.

With steady_clock and duration_cast, we have everything we need for accurate timing.

Part 2: Concurrent Execution with std::thread

Our benchmark design requires two threads running simultaneously: a producer and a consumer. The <thread> library provides the C++ standard way to create and manage threads.

The basic pattern is:

1. Define a function or a lambda that contains the code you want to run in a new thread.
2. Create a std::thread object, passing it the function and any arguments.
3. The new thread starts executing immediately.
4. The main thread can continue its work, but it must eventually call join() on the thread object. This makes the main thread wait until the new thread has finished its execution.

The example code in the README of the SPSC queue repository you've been working with provides a good structural template for a producer-consumer setup using std::thread.

Part 3: Assembling the Benchmark Code

Now, let's combine std::chrono and std::thread to implement the benchmark we designed. We'll build a complete program that:

1. Sets up our SPSC queue.
2. Spawns a producer thread to push a large number of items.
3. Spawns a consumer thread to pop those items.
4. Times the entire process from start to finish.
5. Calculates and prints the throughput in millions of operations per second (MOps/s).

Preventing Unwanted Compiler Optimizations

As we discussed in the last lesson, the compiler might try to outsmart us by optimizing away the work we're trying to measure. A common technique to prevent this is to use a "black box" function that the compiler can't see through. We'll also use volatile on the consumer's result to signal to the compiler that this value can change in ways it cannot predict, forcing it to perform the read.

The Full Benchmark Implementation

Here is the complete C++ code for our benchmark. It follows the design from the previous lesson and uses the techniques discussed today. I've added comments to explain each part of the process.

#include <iostream>
#include <thread>
#include <chrono>
#include <vector>
#include <numeric>
#include <atomic>

// Include the header for the SPSC queue you built in Module 5.
// Make sure this path is correct for your project setup.
#include "LockFreeSpscQueue.h"

// A 'black box' function to prevent the compiler from optimizing away reads.
// This is a common technique in benchmarking.
template<typename T>
void black_box(T&&) {
    asm volatile("");
}

int main() {
    // --- 1. Benchmark Configuration ---
    const size_t queue_capacity = 8192; // Must be a power of two for our queue.
    const size_t total_ops = 100'000'000; // 100 Million operations.

    // --- 2. Setup the Queue ---
    // The data buffer for the queue.
    auto data_buffer = std::vector<uint64_t>(queue_capacity);
    // The queue itself, which manages our buffer.
    LockFreeSpscQueue<uint64_t> spsc_queue(data_buffer);

    // This atomic flag will signal the consumer when to start checking for items.
    std::atomic<bool> start_signal = false;

    // --- 3. Define Producer and Consumer Logic (as lambdas) ---

    // The producer thread's job.
    auto producer_task = [&]() {
        // Wait for the main thread to give the start signal.
        while (!start_signal.load(std::memory_order_relaxed)) {}

        for (uint64_t i = 0; i < total_ops; ++i) {
            // Spin until the push succeeds. `try_write` is non-blocking.
            // Using a lambda that takes a span to write into.
            while (spsc_queue.try_write(1, [&](std::span<uint64_t> block1, std::span<uint64_t> block2){
                // In this single-item write, only block1 will be used.
                block1[0] = i;
                return 1; // We wrote 1 item.
            }) == 0) {
                // Yield the CPU to the consumer thread if the queue is full.
                std::this_thread::yield();
            }
        }
    };

    // The consumer thread's job.
    auto consumer_task = [&]() {
        // This volatile variable ensures the compiler doesn't optimize away our work.
        volatile uint64_t sum = 0;
        uint64_t items_received = 0;

        // Wait for the start signal.
        while (!start_signal.load(std::memory_order_relaxed)) {}

        while (items_received < total_ops) {
            // Spin until the pop succeeds. `try_read` is non-blocking.
            items_received += spsc_queue.try_read(queue_capacity, [&](std::span<const uint64_t> block1, std::span<const uint64_t> block2){
                size_t items_read = 0;
                for (const auto& val : block1) {
                    sum += val; // Do some work with the value.
                    items_read++;
                }
                for (const auto& val : block2) {
                    sum += val;
                    items_read++;
                }
                return items_read;
            });
            
            if (items_received < total_ops) {
                 // Yield if the queue was empty.
                std::this_thread::yield();
            }
        }
        // Pass the final result to the black box to ensure it's "used".
        black_box(sum);
    };

    // --- 4. Run the Benchmark ---
    std::cout << "Starting benchmark with " << total_ops << " operations...\n";

    std::thread producer_thread(producer_task);
    std::thread consumer_thread(consumer_task);

    auto start_time = std::chrono::steady_clock::now();
    start_signal.store(true, std::memory_order_relaxed); // GO!

    // Wait for both threads to complete their work.
    producer_thread.join();
    consumer_thread.join();

    auto end_time = std::chrono::steady_clock::now();

    // --- 5. Calculate and Report Results ---
    auto duration_ns = std::chrono::duration_cast<std::chrono::nanoseconds>(end_time - start_time).count();
    double duration_s = duration_ns / 1e9;
    double mops_per_second = total_ops / duration_s / 1e6;

    std::cout << "Benchmark finished.\n";
    std::cout << "Total time: " << duration_s << " seconds\n";
    std::cout << "Throughput: " << mops_per_second << " MOps/s\n";

    return 0;
}

A Critical Reminder: Compile in Release Mode!

A benchmark is only meaningful when run on optimized code. Debug builds contain extra checks, symbols, and intentionally disabled optimizations that make the code orders of magnitude slower. Always compile your benchmark with optimizations enabled (e.g., -O2, -O3, or the "Release" configuration in your IDE).

Conclusion

Congratulations! You have now implemented a complete, functional microbenchmark for your SPSC queue. You've translated a robust design into practice, using standard C++ libraries for threading and high-precision timing while accounting for tricky compiler optimizations. Running this program will give you a concrete number representing the performance of a critical component of your final project.

Key Takeaways:

• Use std::chrono::steady_clock to measure time intervals for benchmarking, as it is not affected by system time changes.
• Use std::thread to create concurrent producer and consumer tasks that simulate heavy load on the queue.
• The benchmark's main thread should start the timer just before signaling the worker threads to begin and stop it only after both have join()ed.
• Prevent the compiler from optimizing away your benchmarked work by using volatile variables or "black box" functions.
• Always compile benchmarks in Release mode to measure the performance of optimized code.

What's Next?

We've now successfully designed and implemented a test for our queue's throughput. The next logical step is to measure the performance of the instrumentation itself. In the next lesson, you will design a benchmark to measure the overhead of the instrumentation macros, which is essential for proving that our telemetry engine has the "zero-overhead" characteristic we're aiming for.