Hello! Welcome back to our bootcamp.
In our last lesson, we drew a clear line between spinlocks and mutexes, establishing the critical rule: never sleep while holding a spinlock. We saw that this makes spinlocks the only choice for atomic contexts like interrupt handlers. We also briefly touched upon the deadlock danger when a task holding a spinlock is interrupted by an ISR that wants the same lock.
Today, we will dive deep into the canonical solution for this exact problem. This lesson focuses on mastering one of the most important synchronization patterns in kernel development, which is a frequent subject in interviews for senior embedded roles.
Your learning outcome for this lesson is to: Apply IRQ-safe spinlock variants (spin_lock_irqsave) to protect data shared between process context and interrupt context.
1. The Classic Deadlock: Interrupts and Spinlocks
Let's start by revisiting the scenario that makes this topic so critical. In a preemptive, multi-core system, you have two primary execution contexts that can interact with your driver's data:
- Process Context: This is code executing on behalf of a user application, like a
read()orioctl()system call. It can be preempted and is allowed to sleep. - Interrupt Context: This is code executing in an Interrupt Service Routine (ISR) in response to a hardware event. It cannot sleep and typically preempts any process context code running on the same CPU.
The conflict arises when process context holds a lock and an interrupt fires on the same CPU.
The following video from our last lesson on spinlock fundamentals perfectly illustrates this deadlock. Watching it again will set the stage for understanding the Linux-specific solution.
The video 'xv6 Kernel-4: Spinlocks' clearly explains the deadlock that occurs when an interrupt handler tries to acquire a lock already held by the code it interrupted. This is the core problem we are solving today.
Watch from 08:45 to 13:40. Focus on the sequence of events: 1) Thread T acquires a lock. 2) An interrupt preempts Thread T. 3) The interrupt handler tries to acquire the same lock, leading to a deadlock. The video then introduces the general solution: disabling interrupts.
This video demonstrates two key points:
- The race is not just between different CPUs in an SMP system, but also between different execution contexts on the same CPU.
- The fundamental solution is to disable interrupts on the local CPU before acquiring the lock. This ensures an ISR cannot preempt the critical section and try to acquire the same lock.
2. The Solution: spin_lock_irqsave
The Linux kernel provides a specific set of functions designed to implement this "disable interrupts and lock" pattern safely and robustly. The most important of these are spin_lock_irqsave and spin_unlock_irqrestore.
Let's break down exactly what this pair of functions does:
spin_lock_irqsave(spinlock_t *lock, unsigned long flags);
- First, it examines the CPU's interrupt status register and saves its current state into the
flagsvariable you provide. This variable acts as a ticket to remember if interrupts were enabled or disabled before you called the function. - Next, it disables interrupts on the local CPU.
- Finally, it acquires the spinlock, spinning if necessary.
spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags);
- It releases the spinlock, making it available to other threads.
- It then restores the interrupt status on the local CPU using the state saved in your
flagsvariable.
This "save and restore" mechanism is more robust than simply re-enabling interrupts. If your code was called from a context where interrupts were already disabled (e.g., inside another ISR), spin_unlock_irqrestore correctly leaves them disabled. A simpler function that just re-enables them would introduce a bug.
3. Practical Implementation in a Device Driver
Let's apply this to a common device driver pattern: an ISR receives data from hardware and places it into a buffer, while a read() file operation (in process context) consumes data from that buffer.
// Define the shared data and the lock
struct my_device_data {
char circ_buf[1024];
unsigned int head;
unsigned int tail;
// ... other data
};
struct my_device_data g_dev_data;
DEFINE_SPINLOCK(g_dev_lock); // Statically initialize the spinlock
// --- Interrupt Context ---
static irqreturn_t my_irq_handler(int irq, void *dev_id) {
unsigned long flags;
char received_char;
// Acquire lock and disable local interrupts
spin_lock_irqsave(&g_dev_lock, &flags);
// CRITICAL SECTION
received_char = read_hw_register();
g_dev_data.circ_buf[g_dev_data.head] = received_char;
g_dev_data.head = (g_dev_data.head + 1) % 1024;
// END CRITICAL SECTION
// Release lock and restore interrupt state
spin_unlock_irqrestore(&g_dev_lock, &flags);
return IRQ_HANDLED;
}
// --- Process Context ---
static ssize_t my_read(struct file *filp, char __user *buf, size_t len, loff_t *off) {
unsigned long flags;
size_t bytes_to_copy = 0;
// Acquire lock and disable local interrupts
spin_lock_irqsave(&g_dev_lock, &flags);
// CRITICAL SECTION
// Calculate how many bytes can be copied from g_dev_data to the user buffer `buf`
// ...
// Update g_dev_data.tail
// END CRITICAL SECTION
// Release lock and restore interrupt state
spin_unlock_irqrestore(&g_dev_lock, &flags);
// Safely copy data to user space (this can sleep, so it MUST be outside the lock)
if (copy_to_user(buf, /*...source...*/, bytes_to_copy)) {
return -EFAULT;
}
return bytes_to_copy;
}
Notice that both the ISR and the process context code use the exact same _irqsave/_irqrestore pair. This is the standard, safe, and recommended practice.
For a more detailed walkthrough and explanation, the official kernel documentation is an excellent resource.
The kernel documentation, 'Unreliable Guide To Locking', provides authoritative guidance on this topic. This section gives a concrete example of modifying a function to be safe when called from interrupt context.
Read the section titled 'Accessing From Interrupt Context'. It shows a code patch that transforms a function to use spin_lock_irqsave, perfectly illustrating the pattern we just discussed.
4. Interview Deep Dive: Spinlock API Variants
A good follow-up question in an interview is to ask you to compare the different spinlock variants. Your ability to explain the nuances demonstrates a mature understanding of kernel concurrency.
| Function Pair | Disables... | When to Use |
|---|---|---|
spin_lock / spin_unlock |
Nothing (only locks) | For data shared only between process contexts (e.g., two kernel threads). Not safe if an ISR is involved. |
spin_lock_bh / spin_unlock_bh |
Bottom Halves (softirqs, tasklets) | For data shared between process context and a bottom half. This is an optimization; it's less disruptive than disabling all hard IRQs. |
spin_lock_irq / spin_unlock_irq |
Hard Interrupts (IRQs) | For data shared with a hard IRQ handler, but only if you are certain interrupts were enabled before you called it. It blindly re-enables interrupts on unlock. |
spin_lock_irqsave / spin_unlock_irqrestore |
Hard Interrupts (IRQs), saves state | For data shared with a hard IRQ handler. This is the safest and most general variant because it correctly restores the prior interrupt state. |
The following tutorial provides a good breakdown of these different "approaches."
The article 'Spinlock in Linux Kernel 1' from EmbeTronicX provides a practical overview of the different spinlock APIs.
Quickly review 'Approach 3 (Locking between User context and Bottom Halves)', 'Approach 4 (Locking between Hard IRQ and Bottom Halves)', and 'Approach 5 (Alternative way of Approach 4)'. Focus on understanding the purpose of the _bh, _irq, and _irqsave suffixes.
The key takeaway, often repeated by senior kernel developers and summarized in the "Unreliable Guide To Locking", is:
When in doubt, use
spin_lock_irqsave.
It is the most robust and will always do the right thing, even if it's sometimes slightly less performant than a more specialized variant like spin_lock_bh. In an interview, advocating for the safest default is a strong position.
Conclusion
You have now dissected the most critical pattern for safe data sharing between process and interrupt contexts in the Linux kernel. This is a non-negotiable skill for anyone writing device drivers or working deep in the OS.
Key Takeaways:
- Protecting data shared between process context and an ISR requires preventing both multi-core race conditions and same-core deadlocks.
spin_lock_irqsaveis the tool for this job. It acquires a spinlock (for SMP safety) and disables local interrupts (for same-CPU safety).- The
flagsparameter is crucial for_irqsave/_irqrestoreto correctly handle nested locking or calls from contexts where interrupts might already be disabled. - While other variants like
spin_lock_bhandspin_lock_irqexist for specific situations,spin_lock_irqsaveis the most general and safest choice.
Preview of the Next Lesson:
Spinlocks are essential, but they have a performance cost, especially under contention. When one CPU holds a lock, all other CPUs trying to acquire it are spinning and wasting cycles. What if there was a way to allow multiple readers to access data without any locks at all, while still allowing a writer to safely update it? In our next lesson, we will explore a powerful and highly-performant synchronization mechanism that enables this: Read-Copy-Update (RCU).