Understanding Slab Allocators

1. The Need for a Specialized Allocator

Let's start with a simple question: why not just use the buddy allocator for everything? Imagine the kernel needs to allocate memory for a struct file, which is a few hundred bytes. The smallest unit the buddy allocator deals with is a single page, typically 4KB (4096 bytes).

Allocating a 4KB page for a 200-byte object results in over 95% of the memory being wasted. This is known as internal fragmentation. When this happens thousands of times for inodes, process descriptors, and network buffers, the waste becomes immense.

Furthermore, many kernel objects require complex initialization. Constantly allocating a new page, initializing the object from scratch, and then freeing it is computationally expensive. It's far more efficient to keep a "warm" cache of pre-initialized objects ready for immediate use.

To see a great summary of these motivations, let's watch a short video.

This leads us to the solution: a specialized object-caching allocator. In Linux, this is generically known as the slab allocator.

2. Core Concepts: Caches, Slabs, and Objects

The slab allocator acts as a bridge between the page-oriented buddy allocator and the object-oriented needs of the rest of the kernel. The core idea is to create specialized pools of memory for specific object types.

The terminology is crucial:

• Cache: A manager for a single type or size of object. For example, there is a cache for task_struct objects and another for inode objects. Each cache maintains a list of slabs.
• Slab: A contiguous chunk of one or more physical pages, allocated from the buddy system. A slab is the container for objects.
• Object: The actual memory being used by the kernel code. An object is a single chunk of memory within a slab. When a slab is full of objects, it's called a "full" slab. When some objects are in use and some are free, it's a "partial" slab. When all objects are free, it's an "empty" slab.

The basic workflow is as follows:

1. When a kernel subsystem needs an object (e.g., via kmalloc()), it requests it from the appropriate cache.
2. The cache first tries to find a free object in a partial slab. This is the fast path.
3. If no partial slabs are available, it will use an empty slab.
4. If there are no empty slabs either, the cache requests new pages from the buddy allocator to create a new slab, which is then formatted to hold objects.
5. When an object is freed, it is returned to its slab. If this makes the slab transition from full to partial, it's put on the partial list. If it makes the slab empty, the slab may eventually be returned to the buddy allocator to reclaim memory.

3. The Three Implementations: SLOB, SLAB, and SLUB

The term "slab allocator" is generic. The Linux kernel has had three distinct implementations, each with a different design philosophy. The original developer of SLUB, Christoph Lameter, gave an excellent talk explaining the history and technical differences. We'll use it as our guide through this section.

As you just saw, we have to be careful with our terms: a "slab" is a chunk of memory, while "SLAB" is a specific implementation. Now let's dive into the details of each.

Let's summarize and compare the key characteristics based on the talk.

Given that SLUB is the default allocator, we will focus the rest of our lesson on its implementation.

4. A Closer Look at SLUB

SLUB's design achieves both high performance and simplicity by minimizing shared data structures. The key is its heavy reliance on per-CPU data.

Let's break down the most important structures involved.

As the article explains, the most common allocation path is extremely fast and lock-free:

1. The kernel requests an object on a specific CPU.
2. SLUB looks at the kmem_cache_cpu structure for that CPU.
3. This structure points to an "active" slab and a freelist of available objects within that slab.
4. An object is taken from the head of the freelist, and the freelist pointer is updated.
5. This all happens without taking any locks, as the kmem_cache_cpu data is local to that CPU.

Locks are only required in the "slow path," such as when the per-CPU slab becomes full and needs to be exchanged for a partial slab from the shared kmem_cache_node list.

Allocation APIs

Kernel code can interact with the slab allocator in two main ways:

1. General-Purpose Allocation: Using kmalloc() and kfree(). The kernel pre-creates a set of general-purpose caches, named kmalloc-8, kmalloc-16, ..., kmalloc-8k. When you call kmalloc(100), SLUB will find the smallest cache that can fit the request (in this case, kmalloc-128) and allocate an object from it.
2. Special-Purpose Allocation: If a subsystem allocates a specific object type very frequently (e.g., task_struct), it can create its own dedicated cache using kmem_cache_create(). It then uses kmem_cache_alloc() to allocate from this private cache. This improves performance and makes debugging easier by isolating objects.

For a great, step-by-step trace of how a kmalloc(128) call is handled by SLUB, you can refer to the article "Looking at kmalloc() and the SLUB Memory Allocator". It follows the code from the kmalloc inline function down into slab_alloc_node, showing how the correct cache is selected and the per-CPU freelist is updated using an atomic cmpxchg.

5. Practical Inspection and Interview Scenario

You can—and should—inspect the state of the slab allocator on a live system. This is a common activity when debugging performance issues or memory leaks.

Using /proc/slabinfo and slabtop

The /proc/slabinfo file provides a detailed snapshot of all active caches.

$ sudo cat /proc/slabinfo
slabinfo - version: 2.1



# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <shared> : slabdata <active_slabs> <num_slabs> <shared_avail>
...
task_struct       1480   1539   8384    3    8   : tunables    0    0    0 : slabdata    513    513      0
kmalloc-512       40878  43360    512   32    4   : tunables    0    0    0 : slabdata   1355   1355      0
kmalloc-256       21850  21856    256   32    2   : tunables    0    0    0 : slabdata    683    683      0
...

Here you can see:

• name: The name of the cache.
• active_objs: Number of objects currently in use.
• num_objs: Total number of objects (active + free).
• objsize: Size of a single object in bytes.
• objperslab: How many objects fit in one slab.

A more user-friendly, real-time view is provided by the slabtop utility (you may need to install it with sudo apt-get install procps).

Interview Scenario

Your deep understanding of kernel internals is a key asset. An interviewer might ask a question like this to test your practical problem-solving skills:

Question: "You're debugging a driver on a production embedded device. You suspect a use-after-free error is causing random memory corruption, but the system doesn't crash immediately, making it hard to trace. You cannot recompile the kernel or attach a live debugger. How can you use the kernel's built-in memory management features to pinpoint the source of this corruption?"

Conclusion

In this lesson, we bridged the gap between the page-level buddy allocator and the object-level needs of the kernel. By managing pools of small, frequently used objects, the slab allocator system is critical for both performance and memory efficiency.

Key Takeaways:

• Slab allocators solve the internal fragmentation and object initialization overhead issues of using the buddy allocator directly for small allocations.
• The kernel has three implementations: SLOB (for tiny systems), SLAB (the complex predecessor), and SLUB (the modern, scalable, and simple default).
• SLUB achieves high performance by using per-CPU active slabs and freelists, which minimizes locking in the common allocation/deallocation paths.
• SLUB includes powerful built-in debugging features like poisoning and red-zoning that can be enabled at runtime to catch memory corruption bugs.
• You can inspect the state of all slab caches using tools like /proc/slabinfo and slabtop.

You now understand the two primary physical memory allocators in the kernel: the buddy system for pages and the SLUB allocator for small objects. But there's more to the story. What if the kernel needs a large block of memory that is virtually contiguous, but doesn't need to be physically contiguous?

Our next lesson will complete the picture by tackling the learning outcome: "Compare kmalloc, vmalloc, and direct page allocation, and select the appropriate allocator for a given scenario." We will create a clear decision framework for when to use each of the kernel's memory allocation tools.