Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 30

Interrupts & Workqueues: Kernel Module Implementation

Recommended resources

Introduction

Welcome to the next lesson in our exploration of the Linux kernel's inner workings. In the previous lesson, we dissected the three primary deferred work mechanisms: softirqs, tasklets, and workqueues. We established a crucial rule of thumb: if your deferred task needs to sleep (for example, to wait for I/O, allocate memory, or acquire a mutex), you must use a workqueue, as it runs in a safe process context.

Today, we move from theory to practice. This lesson is dedicated to achieving the learning outcome: Implement a kernel module that registers an interrupt handler and defers work using a workqueue. We will build a complete, working kernel module from scratch, demonstrating the classic "top-half/bottom-half" pattern that is fundamental to nearly all Linux device drivers. This hands-on exercise will solidify your understanding and provide a practical template for your future kernel development and interview preparation.

1. The Workqueue API: A Quick Refresher

Before we write code, let's briefly recap the essential components of the workqueue API, which you'll be using.

  • struct work_struct: A structure that represents a piece of work to be done. You will embed this inside your own driver-specific data structure.
  • The Work Handler Function: A function you write that has the signature void my_work_handler(struct work_struct *work). This function contains the code for your "bottom half" and will be executed by a kernel worker thread.
  • Initialization: You initialize your work_struct to point to your handler function. The most common way is with INIT_WORK(&my_work, my_work_handler).
  • Scheduling: From your interrupt handler (the "top half"), you will queue the work using schedule_work(&my_work). This places your work on a shared, system-wide workqueue to be executed as soon as a worker thread is available.

Let's now use these APIs to build our module.

2. Implementing an Interrupt-Driven Workqueue

We will construct our kernel module step-by-step. For this exercise, we don't have a real hardware device to generate interrupts. We will use a clever technique: create a simple character device that, when read, will trigger a software-generated interrupt. This allows us to test our entire interrupt handling path without needing physical hardware.

The following resource provides a great overview and a complete code example that we will study and build upon.

Workqueue in Linux Kernel Part 1

This tutorial from EmbeTronicX, "Workqueue in Linux Kernel Part 1", provides an excellent foundation for our task. It covers the necessary APIs and includes a practical example that we will analyze.

Please read the first two sections, "Bottom Half" and "Workqueue in Linux Kernel". This will reinforce the core concepts from our last lesson: why we need bottom halves and why workqueues are ideal because they run in process context and can sleep.

Step 1: Data Structures and Initialization

First, we need to define a data structure for our work and set up the module's initialization and exit functions. We'll use the static initializer DECLARE_WORK for simplicity in this example.




#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>

/* The interrupt number for the software interrupt we will trigger.
 * On x86, vector 0x3B is unused by default.
 */



#define IRQ_NUM 0x3B 

/* The work handler function prototype */
static void work_handler(struct work_struct *work);

/* Statically initialize a work_struct with our handler */
static DECLARE_WORK(my_work, work_handler);

/* The interrupt handler (top half) prototype */
static irqreturn_t irq_handler(int irq, void *dev_id);

static int __init my_module_init(void) {
    pr_info("Module loaded\n");
    // Registration will happen here
    return 0;
}

static void __exit my_module_exit(void) {
    // Cleanup will happen here
    pr_info("Module unloaded\n");
}

module_init(my_module_init);
module_exit(my_module_exit);
MODULE_LICENSE("GPL");

Step 2: The Bottom Half - The Work Handler

This is where the "real" work gets done. Our function will run in process context, so it's allowed to sleep. To demonstrate this, we will print a message, sleep for a short period, and then print another message.




#include <linux/jiffies.h>
#include <linux/sched.h> // For msleep

/* This is the work handler function (the bottom half) */
static void work_handler(struct work_struct *work) {
    pr_info("Work Handler: Executing in process context of [%s] (PID: %d)\n", 
            current->comm, current->pid);
            
    pr_info("Work Handler: Going to sleep for 20ms...\n");
    msleep(20); // Simulate a long-running, blocking operation
    pr_info("Work Handler: Woke up!\n");
}

Notice we are printing current->comm (the process name) and current->pid. You will see that this code runs in the context of a kernel worker thread, like kworker/uX:Y.

Step 3: The Top Half - The Interrupt Handler

The interrupt handler must be fast and non-blocking. Its only job is to acknowledge the interrupt and schedule the workqueue to run later.

/* This is the interrupt handler (the top half) */
static irqreturn_t irq_handler(int irq, void *dev_id) {
    pr_info("Interrupt Handler: IRQ %d received! Scheduling work.\n", irq);
    
    /* Schedule our work to be executed by a worker thread */
    schedule_work(&my_work);
    
    return IRQ_HANDLED;
}

This function is simple by design. It prints a message to the kernel log and immediately calls schedule_work(). That's it. All the time-consuming logic is deferred to work_handler.

Step 4: Registration and Triggering

Now, let's tie everything together in our module's init function. We need to register our interrupt handler. The request_irq() function asks the kernel to associate our irq_handler with IRQ_NUM.

To trigger the interrupt, the example from EmbeTronicX uses a character device. For simplicity here, we can trigger the software interrupt directly within our init function to see the flow.

static int __init my_module_init(void) {
    pr_info("Module loaded\n");
    
    /* Register our interrupt handler */
    if (request_irq(IRQ_NUM, irq_handler, IRQF_SHARED, "my_irq_device", (void *)(irq_handler))) {
        pr_err("Failed to register IRQ %d\n", IRQ_NUM);
        return -EIO;
    }
    pr_info("IRQ %d registered successfully\n", IRQ_NUM);
    
    pr_info("Triggering software interrupt to test the handler...\n");
    /* On x86, this generates a software interrupt for vector 0x3B */
    asm("int $0x3B");

    return 0;
}

Note on IRQF_SHARED: We use this flag because other devices might theoretically use this interrupt line. When using shared interrupts, you must provide a unique dev_id (we use a pointer to our handler function here) so the kernel can correctly remove your specific handler later.

Step 5: Proper Cleanup

When our module is unloaded, we must release the resources we've acquired. This means unregistering the interrupt handler and ensuring any pending work is cancelled and finished.

Workqueue in Linux Kernel Part 1

Proper cleanup is critical to avoid system instability. Let's look at the functions available for managing pending work items before we write our exit function.

Read the sections "Delete work from workqueue" and "Cancel Work from workqueue". Pay attention to cancel_work_sync(). This function is exactly what we need: it prevents a pending work item from running and waits for an already-running item to complete.

Here is our exit function using these cleanup APIs:

static void __exit my_module_exit(void) {
    /* Wait for the work to finish before proceeding.
     * This is crucial to prevent race conditions on unload.
     */
    cancel_work_sync(&my_work);

    /* Unregister the interrupt handler */
    free_irq(IRQ_NUM, (void *)(irq_handler));
    
    pr_info("IRQ %d freed and module unloaded\n", IRQ_NUM);
}

The order here is important. We first ensure the work handler is no longer running or pending with cancel_work_sync(). Only then is it safe to free the interrupt handler with free_irq().

3. The Complete Code and Testing

Putting it all together, you have a complete module. For the full source code, including the Makefile and the more advanced character device trigger used in the EmbeTronicX example, you can refer to their repository.

Workqueue in Linux Kernel Part 1

The EmbeTronicX tutorial provides the full source code for a slightly more advanced version of this module, which uses a character device to trigger the interrupt on-demand.

Review the "Programming" section and follow the link to GitHub to see the complete source files (driver.c and Makefile). Compile the module using make, load it with sudo insmod driver.ko, and then trigger the interrupt with sudo cat /dev/etx_device.

After loading the module and triggering the interrupt, use the dmesg command. You should see output similar to this, clearly showing the sequence of events:

[ ... ] Module loaded
[ ... ] IRQ 59 registered successfully
[ ... ] create_proc_entry created successfully
[ ... ] etx_open function called
[ ... ] Triggering Interrupt
[ ... ] Interrupt Handler: IRQ 59 received! Scheduling work.
[ ... ] etx_release function called
[ ... ] Work Handler: Executing in process context of [kworker/0:1] (PID: 34)
[ ... ] Work Handler: Going to sleep for 20ms...
[ ... ] Work Handler: Woke up!

This log trace is the proof of our success. The interrupt handler (top half) executed immediately, and the work handler (bottom half) executed shortly after in the context of a kworker thread, where it was able to sleep safely.

4. Interview Scenario

Let's put this into the context of an interview.

Scenario: You are writing a driver for a network interface card (NIC). When a packet arrives, the NIC generates an interrupt. Your top-half ISR reads a status register to acknowledge the IRQ and then defers the rest of the work. The bottom half needs to:

  1. Allocate a new memory buffer (an sk_buff) for the networking subsystem.
  2. Use DMA to copy the packet data from the NIC's internal FIFO to this new buffer.
  3. Pass the buffer to the Linux network stack.

A junior engineer on your team suggests using a tasklet for the bottom half to minimize latency. Explain why this is a dangerous design and what the correct implementation should be.

Click for Analysis

  • Analysis of the Flaw: The suggestion to use a tasklet is incorrect and dangerous for two primary reasons:

    1. Allocation can sleep: The first step, allocating memory (e.g., with alloc_skb(), which can call kmalloc), may need to sleep if the system is under memory pressure. Calling a potentially sleeping function from a tasklet (which runs in interrupt context) is illegal and would cause a kernel panic or deadlock.
    2. DMA can block: DMA operations, while asynchronous, often require waiting for completion. Functions that synchronize with DMA hardware can block. This, too, is illegal in a tasklet.

  • The Correct Solution: The only correct mechanism for this bottom half is a workqueue.

    • The top-half ISR should do the absolute minimum: acknowledge the interrupt and disable further interrupts from the device. Then, it should schedule a work item.
    • The work handler function, running in a process context, can safely call alloc_skb() and perform the blocking DMA transfer. This ensures system stability by moving all potentially blocking operations out of the critical interrupt context. This is precisely how modern network drivers are designed.

Conclusion

In this lesson, you have successfully implemented one of the most common and important patterns in Linux kernel development. You have built a module that correctly handles an interrupt by splitting the work between a fast top half and a flexible, safe bottom half using a workqueue.

Key Takeaways:

  • An interrupt handler (top half) must be extremely fast and is forbidden from sleeping. Its main job is to acknowledge the hardware and schedule deferred work.
  • A workqueue handler (bottom half) runs in process context and is the appropriate place for any long-running or potentially blocking operations like I/O, memory allocation, or sleeping.
  • The core API functions are request_irq(), free_irq(), DECLARE_WORK/INIT_WORK, schedule_work(), and cancel_work_sync().
  • Proper cleanup on module exit is non-negotiable to prevent system instability. Always ensure pending work is finished before freeing the resources it uses.

In our next lesson, we will build on this foundation by exploring what happens when things go wrong. We will learn how to diagnose an interrupt storm scenario, identifying root causes and mitigation strategies.