Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 60

Mutex vs. Spinlock: Performance for Highly Contended Critical Sections

Recommended resources

Hello! Welcome to your next lesson on performance analysis and optimization.

In our last session, we focused on a crucial optimization technique: reducing lock hold time by moving work outside the critical section. We established that the shorter the critical section, the lower the contention and the better the performance. This naturally leads to a question: for these now extremely short critical sections, is a pthread_mutex, with its overhead of potentially putting a thread to sleep and waking it up, still the best tool?

Today, we will explore an alternative: the spinlock. By the end of this lesson, you will be able to compare the performance of a mutex versus a spinlock for a highly contended, short critical section, understanding both the "how" and the "why" of their performance characteristics.

Sleep vs. Spin: The Fundamental Difference

At their core, mutexes and spinlocks solve the same problem—mutual exclusion—but they employ opposite strategies when a lock is already taken.

  • A Mutex sleeps. When a thread tries to acquire a locked mutex, it yields the CPU and asks the operating system's scheduler to put it into a sleeping state. When the lock is released, the scheduler wakes the thread up. This is CPU-efficient but incurs the overhead of two context switches.
  • A Spinlock spins. When a thread tries to acquire a locked spinlock, it enters a "busy-wait" loop, repeatedly checking the lock's status until it becomes available. This avoids context switches but consumes 100% of a CPU core while waiting.

This image provides a clear visual breakdown of the two approaches.

Mutex vs. Spinlock Behavior Comparison
On the left, Thread T2 goes to sleep when it can't acquire the mutex held by T1, freeing up its CPU. On the right, T2 continuously spins, checking the lock held by T1 and consuming its CPU core in the process.

Under the Hood: Kernel Syscalls vs. Userspace Atomics

The "sleep vs. spin" behavior is a direct result of their underlying implementation. Your computer science background will make these distinctions clear.

  • Mutexes (Kernel-Assisted): On Linux, pthread_mutex_t is typically built on futex (Fast Userspace Mutex). A thread attempting to lock a contended mutex makes a futex(FUTEX_WAIT) syscall to go to sleep. The thread that unlocks the mutex makes a futex(FUTEX_WAKE) syscall to wake up a waiting thread. These syscalls and the subsequent context switches are relatively expensive operations (in the order of microseconds). It's worth noting that modern mutexes have a fast path: they first try a cheap atomic operation in userspace and only resort to the expensive syscall if there's contention.

  • Spinlocks (Userspace-Only): A spinlock is implemented entirely in userspace. The "spinning" is a tight loop that uses a single, atomic CPU instruction like compare-and-swap (CAS) to test and acquire the lock. In C, you can implement this with C11's _Atomic types or legacy built-ins. Because there are no syscalls, the latency to acquire an uncontended or briefly-contended lock is extremely low (in the order of nanoseconds).

This fundamental implementation difference is the source of their performance trade-off. To dig deeper into these mechanics, please read the following sections from the article "Spinlocks vs. Mutexes".

Spinlocks vs. Mutexes: When to Spin and When to Sleep

This article, 'Spinlocks vs. Mutexes: When to Spin and When to Sleep' from howtech.substack, provides an excellent, technically deep dive into the mechanisms we've just discussed.

Please read the first two sections: 'The Problem: Every Lock Costs You Something' and 'What Most Articles Don’t Tell You'. Focus on understanding the costs associated with each lock type: syscalls and context switches for mutexes versus CPU burn and cache line bouncing for spinlocks.

The Break-Even Point: When to Spin and When to Sleep

The choice between a spinlock and a mutex comes down to a simple calculation: is the expected wait time for the lock longer or shorter than the overhead of a context switch?

  • If Wait Time < Context Switch Overhead: A spinlock is superior. The CPU cycles spent spinning are fewer than the cycles that would be wasted on sleeping and waking up. This is typical for critical sections that are extremely short (e.g., a few dozen instructions).
  • If Wait Time > Context Switch Overhead: A mutex is superior. The CPU cycles burned by spinning would be enormous. It's far more efficient to let the thread sleep and allow another thread to do useful work on that CPU core.

The "Spinlocks vs. Mutexes" article provides some excellent rules of thumb based on this principle.

Spinlocks vs. Mutexes: When to Spin and When to Sleep

Let's revisit the same article to see some concrete quantitative guidelines for making this choice.

Read the section titled 'The Math That Determines Your Choice'. This section synthesizes the concepts into a practical decision framework based on critical section duration and contention level.

A Practical Benchmark

Theory is useful, but as you prefer, let's get practical. We will now build and analyze two programs that demonstrate these behaviors. One will use a spinlock, the other a mutex, both protecting a short, highly contended critical section.

The following resource provides complete C code for both tests and instructions on how to analyze them. This is the core practical exercise for this lesson.

Spinlocks vs. Mutexes: When to Spin and When to Sleep

The article 'Spinlocks vs. Mutexes' provides two C programs, spinlock_test.c and mutex_test.c, designed specifically to highlight the performance differences. We will use these to conduct our own experiment.

Please read from the section 'Building Your Own Test' to the end of the article. Your goal is to replicate this experiment. Code: Create the two files, spinlock_test.c and mutex_test.c, using the code provided. Notice the spinlock is implemented with atomic_compare_exchange_weak, and the critical section work is simulated with a tiny loop (for (volatile int j = 0; j < 10; j++);). Compile: Compile both programs as instructed: gcc -Wall -Wextra -O2 -pthread <filename.c> -o <exename>. Run & Observe CPU: Run ./spinlock_test and, in another terminal, use top or htop to observe the CPU usage. You should see it spike. Then do the same for ./mutex_test and note the significantly lower CPU usage. Analyze Syscalls: Use strace -c ./spinlock_test and strace -c ./mutex_test to count the syscalls. Note the high number of futex calls for the mutex version and their near-total absence for the spinlock version. (Optional) Analyze Context Switches: Use the cat /proc/$(pgrep ...)/status | grep ctxt command as shown to see the difference in voluntary context switches. Experiment: As the article suggests, try modifying NUM_THREADS and the critical section hold time (by changing the inner loop). Observe how spinlock performance degrades much more rapidly as the hold time increases.

This hands-on experiment will give you a visceral understanding of the trade-offs that no amount of text can fully convey. You'll see the CPU meter hit 100% and know why. You'll see the thousands of futex calls and understand the cost.

Important Caveats and Dangers of Spinlocks

While spinlocks can provide a significant performance boost in the right niche, they are a specialized tool and can be dangerous if misused.

  1. Priority Inversion: If a low-priority thread acquires a spinlock, and then a high-priority thread needs it, the high-priority thread will start spinning. Because it is high-priority, it may hog the CPU, preventing the low-priority thread from ever running and releasing the lock. This is a form of deadlock. pthread_mutex has mechanisms (priority inheritance) to combat this; spinlocks do not.

  2. Preemption: If a thread holds a spinlock and the OS scheduler decides to preempt it (its time slice runs out), any other threads waiting for that lock will now spin uselessly for the entire duration of the preemption, wasting a massive amount of CPU time. This is why spinlocks are generally only recommended on non-preemptible, real-time systems or when the number of threads is less than or equal to the number of CPU cores.

  3. Correctness: Spinlocks cannot be used with condition variables. This makes them unsuitable for many complex synchronization patterns, like the producer-consumer problem. Furthermore, implementing logic around them can be tricky. A common mistake is to check a condition outside the lock, which creates a race condition, as pointed out in this Stack Overflow discussion.

Why are pthread spinlocks so much faster than mutexes?

This Stack Overflow thread, 'Why are pthread spinlocks so much faster than mutexes?', contains an excellent answer that highlights a subtle but critical implementation mistake when using spinlocks for more complex logic.

Briefly read the answer by John Bollinger. You don't need to dive deep into the asker's specific queue code, but focus on Bollinger's key point: to correctly check a condition (like if (me->queue->n_elements > 0)), the spinlock must already be held. This is a great example of the extra care required when working with spinlocks.

Conclusion

You have now analyzed the deep-seated differences between mutexes and spinlocks, from their conceptual models down to their kernel and CPU-level implementations. You've seen firsthand how their performance profiles diverge under contention.

Key Takeaways:

  • Mutexes (Sleep): The general-purpose, safe default. They use kernel syscalls (futex) to sleep when contended, which is CPU-efficient but incurs context switch latency. Best for all but the most specialized cases.
  • Spinlocks (Spin): A specialized optimization for very short, highly contended critical sections. They use userspace atomic instructions to busy-wait, avoiding context switch overhead at the cost of burning CPU cycles.
  • The Litmus Test: The choice depends on whether the lock hold time is shorter than the context switch overhead. This requires measurement and profiling, using tools like top, strace, and perf.
  • Use with Caution: Spinlocks are prone to priority inversion and performance collapse if the lock is held while a thread is preempted. They are not a drop-in replacement for mutexes.

Preview of the Next Lesson:
We've now compared a general-purpose lock (mutex) with a lock for short critical sections (spinlock). Next, we will explore another dimension of optimization. Many data structures are read far more often than they are written. A standard mutex treats readers and writers identically, creating a bottleneck. In our next lesson, we will compare the performance of a mutex versus a read-write lock for a workload with a high read-to-write ratio.