Cgroups v2: Resource Management

1. The Cgroups v2 Architecture: A Unified Approach

Control Groups solve the problem of managing and distributing system resources among collections of processes. The first version, cgroups v1, was powerful but had design issues that made it complex to manage. Cgroups v2 was designed to resolve these issues with a cleaner, more consistent model.

The most significant change in v2 is the move to a single, unified hierarchy. In v1, you could have multiple, independent hierarchies for different resource controllers (e.g., one for CPU, another for memory). This often led to confusion and complexity. Cgroups v2 enforces a single tree structure, which simplifies management and makes the relationship between parent and child groups clear and consistent.

To get a solid introduction from a leading expert, let's watch the beginning of a talk by Michael Kerrisk, the maintainer of the Linux man-pages project.

To summarize the key points:

• Core Idea: Cgroups organize processes hierarchically and distribute system resources along that hierarchy.
• Architecture: It consists of the cgroup core for managing the hierarchy and controllers for managing specific resources (CPU, memory, I/O, etc.).
• Unified Hierarchy: In v2, all controllers reside in a single filesystem tree, typically mounted at /sys/fs/cgroup. This ensures that the process organization is the same for all resource types.
• Interface: All interactions with cgroups happen through standard filesystem operations (mkdir, rmdir, echo, cat) on the pseudo-filesystem.

For a formal definition, the official kernel documentation is the authoritative source. It's a great habit to refer to this, especially if you plan to contribute to open-source projects.

2. Hands-On: Interacting with the Cgroup Filesystem

Theory is important, but true understanding comes from practice. Let's see how the cgroup filesystem works by creating a group, adding a process to it, and applying a simple limit. We will use the pids controller, which limits the number of processes that can exist within a cgroup. This is a great example because its effect is immediate and easy to observe.

The following video segment provides a live demonstration of these steps. I encourage you to open a terminal (with superuser privileges, e.g., sudo -s) and follow along.

Let's break down the core workflow you just observed:

1. Navigate to the cgroup root: cd /sys/fs/cgroup
2. Create a new cgroup: This is just a directory.

   
   
   
   # mkdir my_group
   

3. Move a process into the cgroup: Write the process ID (PID) to the cgroup.procs file. We can use $$ to get the PID of the current shell.

   
   
   
   # echo $$ > my_group/cgroup.procs
   

4. Set a limit: Write a value to the controller-specific file.

   
   
   
   # echo 5 > my_group/pids.max
   

5. Test the limit: Any new process created by the shell (which is now in my_group) will be subject to the cgroup's limits. Trying to create more processes than the limit will fail.

   
   
   
   # for i in $(seq 1 5); do sleep 120 & done
   # bash: fork: retry: Resource temporarily unavailable
   

6. Clean up: To remove a cgroup, you must first ensure it contains no processes and no child cgroups. Then you can simply remove the directory.

   
   
   
   # exit  # (Exit the shell inside 'my_group' to empty it)
   # rmdir my_group
   

This simple exercise reveals the fundamental mechanics of cgroup management through the filesystem.

3. Applying Resource Limits: CPU, Memory, and I/O

Now we get to the core of resource management. To apply a limit for a resource like CPU, you must first enable its controller.

Enabling Controllers

In cgroups v2, resource control is hierarchical. A parent cgroup must explicitly enable controllers for its children. This is done using two key files:

• cgroup.controllers: Shows which controllers are available to be enabled.
• cgroup.subtree_control: A read/write file where you enable/disable controllers for child cgroups. You enable a controller by writing its name prefixed with + (e.g., echo '+cpu +memory' > cgroup.subtree_control).

This next video segment explains this crucial concept.

Now, let's apply this to our three target resources.

A. CPU Controller

The CPU controller manages the distribution of CPU time. It provides two main modes of control:

• Proportional Weighting (cpu.weight): Distributes CPU time proportionally among competing cgroups. If group A has a weight of 100 and group B has a weight of 200, group B will get twice as much CPU time as group A when they are both busy. Default is 100.
• Absolute Bandwidth Limit (cpu.max): Sets a hard cap on CPU usage. It's defined by a quota and a period, both in microseconds. For example, echo "50000 100000" means the cgroup can use 50,000µs (0.05s) of CPU time every 100,000µs (0.1s), effectively limiting it to 50% of one CPU core.

Practical Exercise: Limiting CPU

1. Create a test group: mkdir /sys/fs/cgroup/cpu_test
2. Enable the cpu controller in the parent (root cgroup): echo '+cpu' > /sys/fs/cgroup/cgroup.subtree_control.
3. Set a 20% CPU limit on your new group: echo "20000 100000" > /sys/fs/cgroup/cpu_test/cpu.max.
4. In a separate terminal, start a CPU-consuming task, like sha1sum /dev/zero. Note its PID.
5. Move the task into the cgroup: echo <PID> > /sys/fs/cgroup/cpu_test/cgroup.procs.
6. Observe the CPU usage with a tool like top or htop. You should see it drop to and hold at 20%.

B. Memory Controller

The memory controller is more complex because memory is a stateful resource. Key interface files include:

• memory.max: A hard limit on memory usage. If a process in the cgroup tries to allocate memory beyond this limit (and the kernel can't reclaim memory from the cgroup), the Out-Of-Memory (OOM) killer will be invoked on a process within that cgroup.
• memory.high: A soft limit. When usage exceeds this, the kernel will heavily throttle the processes, forcing them to reclaim memory. It does not invoke the OOM killer, making it a safer way to control memory bloat.
• memory.low: A best-effort protection. The kernel tries to avoid reclaiming memory from a cgroup that is below this boundary.
• memory.swap.max: Limits the amount of swap space the cgroup can use.

Practical Exercise: Limiting Memory

1. Create a test group: mkdir /sys/fs/cgroup/mem_test.
2. Enable the memory controller: echo '+memory' > /sys/fs/cgroup/cgroup.subtree_control.
3. Set a 100MB memory limit: echo 100000000 > /sys/fs/cgroup/mem_test/memory.max.
4. Move your current shell into the cgroup: echo $$ > /sys/fs/cgroup/mem_test/cgroup.procs.
5. Run a program that allocates memory. stress-ng is great for this, but a simple Python one-liner also works: python3 -c "a = ' ' * 200 * 1024 * 1024; print('allocated')"
6. The command will likely be terminated by the kernel. You can check for OOM killer messages in the kernel log using dmesg. You should see a message indicating a process was killed to satisfy the cgroup memory limit.

C. I/O Controller

The I/O controller regulates bandwidth to block devices (e.g., hard drives, SSDs). Like the CPU controller, it provides both weighted and absolute limits.

• io.weight: Proportional I/O time sharing.
• io.max: Sets maximum read/write bandwidth (in bytes per second) or IOPS (I/O operations per second).

Practical Exercise: Limiting I/O

1. Find your root disk's major and minor device numbers: ls -l /dev/sda (or nvme0n1, etc.). For example, /dev/sda might be 8, 0.
2. Create a test group: mkdir /sys/fs/cgroup/io_test.
3. Enable the io controller: echo '+io' > /sys/fs/cgroup/cgroup.subtree_control.
4. Set a 1 MB/s write limit for your disk (replace 8:0 with your device numbers): echo "8:0 wbps=1000000" > /sys/fs/cgroup/io_test/io.max.
5. Move your shell into the cgroup: echo $$ > /sys/fs/cgroup/io_test/cgroup.procs.
6. Perform a disk-intensive write operation. dd is the classic tool. Note the transfer speed.

   
   
   
   # dd if=/dev/zero of=testfile bs=1M count=100 oflag=direct
   

   The oflag=direct helps bypass the page cache to make the I/O limit more immediately apparent. You should see the speed reported by dd hover around 1 MB/s. Don't forget to rm testfile afterwards.

4. Interview Scenario

Your experience with resource contention in embedded systems makes cgroups a powerful tool in your arsenal.

Scenario: You're designing the software for an automotive infotainment system. It runs two main applications: a critical navigation service and a less critical media player app. The system must guarantee that the navigation service is always responsive, even if the media player is performing a heavy operation like indexing a large music library from a USB drive.

Question: How would you leverage cgroups v2 to architect a solution that ensures the navigation service's performance is protected from the media player?

Conclusion

In this lesson, we bridged the gap from resource visibility (namespaces) to resource control (cgroups). You now have a complete foundational understanding of the two pillars of Linux containerization.

Key Takeaways:

• Unified Architecture: Cgroups v2 uses a single, hierarchical tree to manage all resources, which simplifies administration.
• Filesystem Interface: All management is done through file and directory operations in /sys/fs/cgroup.
• Hierarchical Control: Controllers must be enabled in a parent's cgroup.subtree_control file to manage resources for its children.
• Applying Limits: You can apply both proportional (.weight) and absolute (.max) limits for resources like CPU, memory, and I/O to build robust, predictable systems.

Preview of the Next Lesson:

We have now seen how to control what a process can see (namespaces) and how much of a resource it can use (cgroups). The final piece of the isolation puzzle is controlling what a process can do. In our next lesson, we will explore the seccomp-bpf mechanism, which allows you to restrict the specific system calls a process is permitted to make, providing a powerful layer of security hardening.