Demand Paging and Page Fault Handling in Linux

Introduction

In our last lesson, we navigated the kernel's internal memory allocators—kmalloc, vmalloc, and alloc_pages—focusing on how the kernel acquires memory for its own use. You learned how the choice between them is dictated by the need for physical contiguity, allocation size, and performance, a crucial concept for tasks like DMA.

Today, we shift our focus from the kernel's needs to how it manages memory for user-space processes. When a process calls malloc(), does the kernel immediately find and allocate all the requested physical RAM? The answer is no, and the mechanism it uses instead is one of the most fundamental optimizations in modern operating systems.

Your goal for this lesson is to describe how demand paging and page fault handling work in the Linux kernel. We will explore the "lazy" allocation strategy where physical memory is only assigned at the very last moment—when it's first accessed. Understanding this process, from the hardware trap to the intricate software handling, is essential for diagnosing performance issues, understanding memory usage, and acing advanced interview questions.

1. The Principle of Lazy Allocation: Demand Paging

The core idea behind demand paging is simple yet powerful: don't allocate a physical page frame until a process actually tries to use it.

When a user-space process requests a large chunk of memory (e.g., via malloc() or brk()), the kernel doesn't immediately reserve physical RAM. Instead, it just creates or updates a Virtual Memory Area (VMA), a vm_area_struct in its internal data structures. This VMA is essentially a promise: it records that the process has a valid claim to a certain range of virtual addresses. The corresponding page table entries (PTEs) are left empty or marked as "not present."

The actual physical memory is only allocated when the process attempts to read from or write to an address within that VMA for the first time. This triggers a page fault.

To see how this lazy design philosophy is central to Linux, let's watch a short video.

All Kernel Page Faults Explained

The video "All Kernel Page Faults Explained" from Hussein Nasser provides an excellent starting point by explaining why page faults are an intentional part of the kernel's design.

Watch the segment from 02:26 to 03:40. Focus on the concept of the kernel as a "lazy load by design" and how the "present" bit in the page table entry is the key to this mechanism.

This "lazy" approach has two major benefits:

Faster Process Startup: A program like bash can be loaded instantly because the kernel doesn't need to copy its entire executable into RAM at the start. Pages are loaded from the file only as they are needed.
Efficient Memory Usage: Memory is not wasted on allocations that a process requests but never actually uses.

2. The Page Fault: A Cooperative Effort between Hardware and Software

A page fault is not an error in the traditional sense. It's a hardware-generated exception (a trap) that signals the operating system to intervene and resolve a memory access. The process is a beautifully choreographed dance between the CPU's Memory Management Unit (MMU) and the kernel's page fault handler.

Let's walk through the entire sequence from a high level.

Linux Kernel Internals: Memory Management

The video "Linux Kernel Internals: Memory Management" by Maple Circuit provides a clear, step-by-step visual explanation of the entire page fault process, from the CPU's request to the kernel's resolution.

Please watch the following segments: The MMU and TLB (02:58 - 04:31): Understand the role of the MMU in translating virtual to physical addresses and how the TLB acts as a cache. Note what happens on a TLB miss. The Page Table Walk (04:31 - 07:50): This is critical. Follow how the CPU hardware walks the multi-level page table (PGD, PUD, PMD, PTE) to find the physical address. The fault occurs when this walk fails. Full Process Recap (14:06 - 15:54): This segment ties everything together, reviewing the flow from a CPU request, to a TLB miss, to a page fault, and finally to the kernel resolving the fault by consulting VMAs.

To summarize the flow you just watched:

A user process executes an instruction like MOV [address], eax.
The CPU's MMU receives the virtual address. It first checks its high-speed cache, the Translation Lookaside Buffer (TLB).
TLB Miss: If the translation isn't in the TLB, the MMU must perform a "page walk." It starts from the CR3 register (on x86), which points to the base of the top-level page table (PGD), and traverses the hierarchy down to the final Page Table Entry (PTE).
Page Fault Exception: The MMU finds a PTE that is either invalid or marked as "not present." It cannot complete the translation. It stops, saves the faulting address and error information, and triggers a hardware exception, switching the CPU into kernel mode and transferring control to the kernel's pre-registered page fault handler.

At this point, the hardware's job is done, and the kernel software takes over.

3. Types of Page Faults

The kernel's page fault handler is a complex piece of code because "page not present" can mean several different things. The handler must inspect the faulting address, the VMA that covers it, and the state of the PTE to determine the correct course of action.

These actions fall into a few main categories.

Fault Type	Description	Kernel Action
Minor Fault	The page was accessed for the first time (demand allocation). The data doesn't exist yet or is a page of all zeros.	Allocate a new physical page, map it in the PTE, and resume the process. This is a relatively fast operation.
Major Fault	The page exists but has been temporarily moved from RAM to a backing store (like a swap partition or file on disk).	Find the data on disk, read it into a newly allocated physical page, map the page, and resume the process. This is slow due to I/O.
Copy-on-Write (COW) Fault	A process attempts to write to a shared page that is currently marked read-only.	Create a private copy of the page for the writing process, map the new page as writable in its PTE, and resume.
Invalid Access	The process tried to access an address for which it has no valid VMA, or it violated permissions (e.g., writing to a read-only code segment).	This is a true error. The kernel sends a signal, typically `SIGSEGV` (segmentation fault), to the process, which usually terminates it.

The following video gives excellent real-world examples for each of these scenarios.

All Kernel Page Faults Explained

Let's return to Hussein Nasser's "All Kernel Page Faults Explained" to see concrete examples of when different types of faults occur.

Watch the following distinct scenarios: First Access (05:11 - 08:08): This covers the classic minor fault for an anonymous page. Copy on Write (09:13 - 11:47): Explains how fork() uses COW and how a write triggers a fault to create a private copy. Swap (12:42 - 14:16): Describes a major fault where a page must be read back from the swap file. Permissions (15:11 - 18:20): Shows how a permission violation (like trying to execute from the stack) also causes a page fault, but one that leads to an error.

4. Inside the Page Fault Handler: A Code-Level Look

Now that you have the high-level concepts, let's trace the execution path inside the kernel. Your experience with embedded software and C will be valuable here as we connect the concepts to actual kernel functions. We will focus on the most common case: a minor fault on an anonymous memory page (e.g., the first write after a malloc).

The journey from the hardware trap to a resolved fault is intricate. The following article provides a superb, detailed walkthrough of the kernel source code.

mapping anonymous memory into a user process's address space

The article "mapping anonymous memory into a user process's address space" by Ryan Stenberg traces the exact functions called during a page fault for anonymous memory on x86. This provides the deep, code-level insight you're looking for.

Read the article from the "Code Investigation" section to the end. As you read, focus on this chain of events: The Entry Point: The fault starts in architecture-specific code (exc_page_fault) and quickly calls a generic C function (handle_page_fault). Finding the VMA: The kernel validates the access by calling find_vma to ensure the address falls within a legitimate memory region for the process. Allocating Page Directories: The __handle_mm_fault function walks the page table hierarchy. If intermediate directories (PGD, P4D, etc.) are missing, it allocates them. Handling the PTE: handle_pte_fault is called. For an anonymous page with no PTE, it calls do_anonymous_page. The Final Steps: Inside do_anonymous_page, a physical page is allocated (alloc_zeroed_user_highpage_movable), a valid PTE is created (make_pte), and it's atomically placed into the page table (set_pte_at).

The diagram below provides a visual summary of the decision logic inside the kernel's fault handler. You can see the paths for different fault types, such as demand paging, copy-on-write, and the invalid access that results in a SIGSEGV.

Linux Kernel Page Fault Handling Flowchart — This flowchart illustrates the kernel's decision-making process when handling a page fault. It shows how the kernel distinguishes between valid accesses that need resolution (like demand paging or COW) and invalid accesses that result in an error signal to the process.

5. Interview Scenario

Your deep understanding of this mechanism is prime material for a senior-level interview. Consider the following question:

Question: "A server application allocates a 1GB buffer using malloc, but doesn't write to it immediately. When profiling the system, you observe that the process's virtual memory size (VIRT) is over 1GB, but its resident set size (RES) is only a few megabytes. Later, when the application starts writing sequentially into the buffer, you see a high rate of minor page faults, and the RES value steadily climbs towards 1GB. Explain precisely what is happening at the kernel level."

Click to see the analysis

A comprehensive answer would break down the behavior step-by-step:

Initial State (High VIRT, Low RES): The call to malloc(1GB) doesn't allocate 1GB of physical RAM. The C library communicates with the kernel (likely via the brk or mmap syscall) to create a 1GB vm_area_struct (VMA) in the process's address space. This reserves the virtual address range, hence the high VIRT. However, no physical pages are allocated, and the page table entries for this range are marked as not-present. The RES is low because it only accounts for physical memory currently in use (e.g., for the executable code, stack, and C library).
The Write Operation (Minor Fault Storm): When the application starts writing to the buffer, it touches a series of virtual addresses for the first time.
- For the very first write, the CPU's MMU attempts to translate the virtual address, finds a "not-present" PTE, and triggers a page fault.
- The kernel's page fault handler takes over. It checks that the faulting address is within the valid 1GB VMA created earlier.
- Recognizing this as a legitimate access to an anonymous page, it triggers a minor fault.
- The handler (do_anonymous_page) allocates a single, zeroed physical page frame using the buddy allocator.
- It creates a valid PTE that maps the faulting virtual address to this new physical page frame, setting the present, writable, and dirty bits.
- The handler then returns, and the CPU re-executes the failed instruction, which now succeeds. The process's RES is incremented by 4KB.
The Climbing RES: This process repeats for every new 4KB page the application writes to within the 1GB buffer. Each access to a new page triggers another minor fault, causing the kernel to allocate another physical page. This "storm" of minor faults is the visible effect of demand paging in action, and it explains why the RES value climbs steadily as the buffer is filled.

Conclusion

In this lesson, you have dissected one of the most elegant mechanisms in the Linux kernel. You've seen how demand paging enables the efficient, on-demand use of physical memory, and how the page fault is the central trigger for this process.

Key Takeaways:

Demand Paging is a "lazy" strategy: physical memory is not allocated until a virtual address is first accessed.
A Page Fault is a hardware exception triggered by the MMU when it cannot translate a virtual address, typically because the corresponding page table entry (PTE) is marked "not present".
Faults are not always errors. The kernel's fault handler interprets the fault to perform actions like demand allocation (minor fault), swapping in from disk (major fault), or copy-on-write.
The kernel code path involves validating the access against a VMA, walking and creating page tables, allocating a physical page, and finally creating and installing a valid PTE.

We have seen how crucial vm_area_structs are for managing a process's address space and validating page faults. In our next lesson, we will dive deeper into this topic: "Explain the process address space layout, including VMA structures and how the mmap system call is implemented." You will learn how these VMA structures are created and managed, particularly through the powerful mmap system call, which is the foundation for loading applications, libraries, and sharing memory.

Recommended resources