Mutex vs. R/W Lock Performance for Read-Heavy Workloads

The Problem: Unnecessary Serialization

Imagine a scenario with ten threads. Nine need to read a configuration value, and one needs to update it. With a standard pthread_mutex_t, only one thread can hold the lock at a time. This means the nine reader threads, which could all safely read the data concurrently without interfering with each other, are forced to execute serially.

This image perfectly illustrates the inefficiency:

The Solution: pthread_rwlock_t

The POSIX threads library provides a specialized tool for this exact problem: the read-write lock (pthread_rwlock_t). It allows for a more granular locking policy:

• Shared Read Lock: Multiple threads can acquire a read lock simultaneously.
• Exclusive Write Lock: Only one thread can acquire a write lock at a time. While a write lock is held, no other threads (readers or writers) can acquire any lock.

This allows you to maximize concurrency for readers while still ensuring correctness and safety for writers.

The following video provides a concise introduction to the concept and walks through a C example using the pthreads API.

The Performance Trade-Off: Overhead vs. Parallelism

A read-write lock sounds strictly better, but there's a trade-off. Its internal logic is more complex than a simple mutex. It needs to track the number of active readers, whether a writer is waiting, and manage queues for both. This additional complexity means that the individual lock and unlock operations for a pthread_rwlock_t are more expensive than for a pthread_mutex_t.

Therefore, a read-write lock is beneficial only when the performance gained from running readers in parallel outweighs the higher overhead of the lock itself.

The following video discusses this trade-off and shows benchmark results that quantify it.

As the video demonstrates, the "magic ratio" of readers to writers is what determines whether an rwlock is the right choice. If your workload is mostly writes, the higher overhead of the rwlock can actually make your application slower than if you had used a simple, lower-overhead mutex.

Practical Benchmark: Mutex vs. Read-Write Lock

Now, let's put this to the test. Your task is to write a C program that benchmarks the two lock types under different read/write workloads.

Here is a skeleton to get you started. The goal is to fill in the // TODO sections.

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h>
#include <unistd.h> // For usleep

#define NUM_LOOPS 10000

// Shared data
long long shared_counter = 0;

// TODO 1: Declare both a mutex and a read-write lock
pthread_mutex_t mutex_lock;
pthread_rwlock_t rw_lock;

// A flag to choose which lock to use
int use_rw_lock = 0;

// Reader thread function
void* reader(void* arg) {
    (void)arg; // Unused
    for (int i = 0; i < NUM_LOOPS; ++i) {
        if (use_rw_lock) {
            // TODO 2: Acquire the read lock
            pthread_rwlock_rdlock(&rw_lock);
        } else {
            pthread_mutex_lock(&mutex_lock);
        }

        // Simulate some "read" work inside the critical section
        long long val = shared_counter;
        (void)val; // Suppress unused variable warning
        usleep(1); // Hold the lock for 1 microsecond

        if (use_rw_lock) {
            // TODO 3: Release the read lock
            pthread_rwlock_unlock(&rw_lock);
        } else {
            pthread_mutex_unlock(&mutex_lock);
        }
    }
    return NULL;
}

// Writer thread function
void* writer(void* arg) {
    (void)arg; // Unused
    for (int i = 0; i < NUM_LOOPS; ++i) {
        if (use_rw_lock) {
            // TODO 4: Acquire the write lock
            pthread_rwlock_wrlock(&rw_lock);
        } else {
            pthread_mutex_lock(&mutex_lock);
        }

        // Simulate some "write" work inside the critical section
        shared_counter++;
        usleep(1); // Hold the lock for 1 microsecond

        if (use_rw_lock) {
            // TODO 5: Release the write lock
            pthread_rwlock_unlock(&rw_lock);
        } else {
            pthread_mutex_unlock(&mutex_lock);
        }
    }
    return NULL;
}

int main(int argc, char *argv[]) {
    if (argc != 4) {
        fprintf(stderr, "Usage: %s <num_readers> <num_writers> <use_rw_lock: 0 or 1>\n", argv[0]);
        return 1;
    }

    int num_readers = atoi(argv[1]);
    int num_writers = atoi(argv[2]);
    use_rw_lock = atoi(argv[3]);

    int total_threads = num_readers + num_writers;
    pthread_t threads[total_threads];

    // TODO 6: Initialize the chosen lock
    if (use_rw_lock) {
        pthread_rwlock_init(&rw_lock, NULL);
    } else {
        pthread_mutex_init(&mutex_lock, NULL);
    }

    struct timespec start, end;
    clock_gettime(CLOCK_MONOTONIC, &start);

    // Create reader threads
    for (int i = 0; i < num_readers; ++i) {
        pthread_create(&threads[i], NULL, reader, NULL);
    }

    // Create writer threads
    for (int i = 0; i < num_writers; ++i) {
        pthread_create(&threads[num_readers + i], NULL, writer, NULL);
    }

    // Join all threads
    for (int i = 0; i < total_threads; ++i) {
        pthread_join(threads[i], NULL);
    }

    clock_gettime(CLOCK_MONOTONIC, &end);
    
    // TODO 7: Destroy the chosen lock
    if (use_rw_lock) {
        pthread_rwlock_destroy(&rw_lock);
    } else {
        pthread_mutex_destroy(&mutex_lock);
    }

    double time_taken = (end.tv_sec - start.tv_sec) * 1e9;
    time_taken = (time_taken + (end.tv_nsec - start.tv_nsec)) * 1e-9;

    printf("Lock Type: %s, Readers: %d, Writers: %d\n",
           use_rw_lock ? "RW Lock" : "Mutex", num_readers, num_writers);
    printf("Time taken: %f seconds\n\n", time_taken);

    return 0;
}

Your Experiment:

1. Complete the Code: Fill in the TODO sections. The function names are provided in the video and this lesson text.
2. Compile: gcc -Wall -Wextra -O2 -pthread your_file.c -o benchmark
3. Run and Measure: Execute the program with different configurations. Try to keep the total number of threads constant to make comparisons fair. For example, on a machine with 8 cores, you could test:
   • ./benchmark 8 0 0 (Mutex, 8 readers) vs ./benchmark 8 0 1 (RW Lock, 8 readers)
   • ./benchmark 7 1 0 (Mutex, 7 readers, 1 writer) vs ./benchmark 7 1 1 (RW Lock, 7 readers, 1 writer)
   • ./benchmark 4 4 0 (Mutex, 4 readers, 4 writers) vs ./benchmark 4 4 1 (RW Lock, 4 readers, 4 writers)
   • ./benchmark 1 7 0 (Mutex, 1 reader, 7 writers) vs ./benchmark 1 7 1 (RW Lock, 1 reader, 7 writers)
   • ./benchmark 0 8 0 (Mutex, 8 writers) vs ./benchmark 0 8 1 (RW Lock, 8 writers)

Interpreting Your Results

After running your tests, you should see a clear pattern emerge. For read-heavy workloads, the rwlock version should be substantially faster. As the number of writers increases, this advantage will diminish and eventually disappear. In the write-only case, you might even find the mutex is slightly faster due to its lower overhead.

Your results should align with the trend shown in this benchmark graph:

A Final Detail: Preventing Writer Starvation

What happens if a writer wants the lock, but new readers keep arriving and acquiring read locks? The writer could be "starved" and wait indefinitely. To prevent this, most pthread_rwlock_t implementations give some preference to waiting writers.

The resource below explains this behavior.

Conclusion

You have now explored a crucial optimization technique for concurrent data structures. By understanding the access patterns of your application, you can choose a locking primitive that maximizes performance.

Key Takeaways:

• A pthread_mutex_t serializes all access, which is inefficient for read-heavy workloads where concurrent reads are safe.
• A pthread_rwlock_t provides a more nuanced approach, allowing multiple concurrent readers or a single exclusive writer.
• This performance benefit comes at the cost of higher per-operation overhead for the lock itself.
• The choice is data-driven: A read-write lock is only a net win when the performance gain from parallelizing reads outweighs its higher overhead. This typically requires a high read-to-write ratio and a non-trivial amount of work within the read-side critical section.
• Always benchmark your specific workload to validate your choice of synchronization primitive.

Preview of the Next Lesson:
We've spent this module optimizing how we use locks. We've compared mutexes to spinlocks for short critical sections and to read-write locks for read-heavy workloads. But what if we could get rid of locks entirely for simple operations? In our next lesson, we will begin exploring the world of lock-free programming by comparing the performance of a lock-based versus a lock-free implementation of a shared counter using C11 atomic operations.