Virtual Memory: Address Space, VMAs, and mmap

1. The Process Virtual Address Space Layout

Every process in Linux operates within its own private virtual address space, a conceptual sandbox that isolates it from other processes and the kernel itself. On a 64-bit system, this is a vast 256TB space, typically split between user space (128TB) and kernel space (128TB).

A user-space process isn't just one monolithic block of memory; it's organized into several distinct segments, each serving a specific purpose.

Let's break down the key user-space segments shown in the diagram:

• Text Segment (.text): This is where the executable machine code of your program resides. It's mapped as read-only and executable. If multiple processes run the same program (e.g., /bin/bash), they will all share the same physical pages for this segment.
• Data Segments (.data, .bss):
  • The .data segment holds initialized global and static variables.
  • The .bss segment holds uninitialized global and static variables. The kernel automatically initializes this area to zero, cleverly using a single physical page of zeros for all such pages until they are written to (a concept called a demand-zero page).
• Heap: This is the area for dynamic memory allocation, managed by functions like malloc(), calloc(), and free(). In Linux, the heap is typically managed by growing a memory segment via the brk() system call. It grows upwards from the end of the BSS segment.
• Memory Mapping Segment (mmap region): This flexible area is used for various mappings, including shared libraries, memory-mapped files, and anonymous memory allocations requested via mmap(). It typically starts at a high address and grows downwards.
• Stack: This area stores local variables, function parameters, and return addresses for function calls. It has a fixed maximum size and grows downwards towards the mmap region.

Each of these segments, and indeed any memory region managed by the kernel for a process, is described internally by a vm_area_struct.

2. The vm_area_struct: The Kernel's Blueprint for Memory

The struct vm_area_struct, or VMA, is the fundamental kernel object for managing a process's address space. Each VMA represents a contiguous range of virtual addresses with a uniform set of properties, such as permissions (read, write, execute) and backing source (a file or anonymous memory).

The kernel documentation provides the most authoritative definition.

A single process has many VMAs, one for its code, one for data, one for the stack, several for each shared library, and more for any memory allocated with mmap(). All VMAs for a process are managed within its mm_struct, which is the top-level descriptor for a process's memory. Historically, these were stored in a red-black tree, but modern kernels use a more cache-efficient data structure called a maple tree.

To truly understand VMAs, let's examine their fields.

From your reading, here are the most critical fields to remember:

• vm_start, vm_end: These define the start (inclusive) and end (exclusive) of the contiguous virtual address range covered by the VMA.
• vm_mm: A pointer back to the parent mm_struct, linking the VMA to the process's overall memory space.
• vm_flags: A bitmask defining the properties of this memory region. You'll see flags like VM_READ, VM_WRITE, VM_EXEC, and VM_SHARED. These are checked during a page fault to determine if an access is legal (e.g., preventing a write to a read-only code segment).
• vm_file: If the VMA is backed by a file (e.g., an executable or a memory-mapped data file), this field points to the corresponding struct file. For anonymous memory (like the heap or a malloc via mmap), this is NULL.
• vm_ops: A pointer to a struct vm_operations_struct containing function pointers for operations on this VMA, such as fault() or open(). This allows different types of memory (e.g., regular memory vs. device driver memory) to have custom behavior.

3. Creating Mappings with mmap()

While some VMAs are created implicitly when a program is loaded, the mmap() system call is the primary explicit tool for creating new ones. It is far more versatile than brk() and is the foundation for loading shared libraries, efficient file I/O, and inter-process communication.

mmap() can create two main types of mappings:

1. File-backed Mappings: The virtual memory is associated with a file on disk. When a page is first accessed, the kernel reads the corresponding data from the file into physical memory (a major page fault). This is how executables and shared libraries are loaded.
2. Anonymous Mappings: The virtual memory is not backed by a specific file. It is initialized to all zeros. This is a common way for malloc implementations to get large chunks of memory from the kernel.

Furthermore, mappings can be private or shared:

• MAP_PRIVATE: Writes to the mapping are not visible to other processes and are not propagated to the underlying file. The kernel uses the Copy-on-Write (COW) mechanism we discussed in the last lesson. When a process first writes to a private page, the kernel transparently creates a private copy of that page for the writing process.
• MAP_SHARED: Writes are visible to all processes that have mapped the same object, and for file-backed mappings, the changes are carried through to the underlying file.

This next video provides an excellent conceptual overview of mmap and these different mapping types.

4. The mmap() Implementation: A Kernel Deep Dive

Now that you understand the concepts, let's trace the mmap() system call into the kernel to see how a VMA is actually created. Your experience with embedded C will be very helpful here, as we'll be looking at the function call sequence.

The journey from a user-space mmap() call to a new VMA in the process's maple tree involves several key kernel functions. The following article provides an excellent code-level walkthrough.

To summarize the flow you just read:

1. The mmap() system call enters the kernel, and after some architecture-specific setup, calls the generic do_mmap() function.
2. do_mmap() is the primary validation and preparation layer. It checks permissions, validates flags, and calls get_unmapped_area() to find a sufficiently large gap in the process's virtual address space.
3. Once it has a valid address and flags, do_mmap() calls mmap_region() to perform the mapping.
4. mmap_region() is the core implementation. It tries to be efficient by first checking if the new mapping can be merged with an existing adjacent VMA. If not, it allocates a new vm_area_struct from the SLUB allocator, meticulously initializes its fields (vm_start, vm_end, vm_flags, vm_file, etc.), and finally inserts this new VMA into the process's mm_struct->mm_mt maple tree.

The following diagram shows the complex web of structures involved in memory management. You can now see how the mm_struct links to the list of vm_area_structs, which in turn can point to file and address_space objects for file-backed mappings.

5. Interview Scenario

Let's test this knowledge with a scenario-based question typical of a senior embedded role interview.

Question: "You run two separate instances of a data processing application. Both applications memory-map the same 2 GB data file using mmap() with MAP_PRIVATE and PROT_READ | PROT_WRITE. Process A reads the entire file. Process B then modifies a single byte at a 1GB offset within its mapping. Describe the state of the VMAs and physical memory (1) after Process A reads the file, and (2) after Process B performs its write."

Conclusion

In this lesson, we have constructed a detailed model of a process's virtual address space. You now understand that this space is not arbitrary but is a well-organized collection of Virtual Memory Areas (VMAs), each defined by a struct vm_area_struct.

Key Takeaways:

• A process's virtual address space is segmented into text, data, BSS, heap, mmap region, and stack.
• The kernel uses struct vm_area_struct (VMA) to manage each contiguous region, tracking its address range, permissions (vm_flags), and backing source (vm_file).
• The mmap() system call is the primary interface for creating new VMAs, offering flexible control over file-backed vs. anonymous and shared vs. private (COW) mappings.
• The kernel implementation of mmap() involves do_mmap() for validation and mmap_region() for VMA merging or allocation and insertion into the process's memory map.

So far, we have focused on how memory is promised and mapped. But what happens when the system is under pressure and physical RAM runs low? In our next lesson, we will investigate the page reclamation mechanism, including LRU lists, kswapd, direct reclaim, and the OOM killer, to see how the kernel decides which pages to discard to make room for new ones.