Managing Disk Space with `df` and `du`

1. df: The Filesystem Overview

The first command, df, stands for disk free. It operates at the filesystem level, giving you a quick summary of all mounted storage devices and their usage. Think of it as the 30,000-foot view of your server's storage.

Let's watch a video that introduces the command and its most important option.

As you saw, the output of df -h gives you everything you need for a quick health check. In a data center, you'd scan the Use% column for any filesystems approaching critical levels (e.g., over 85% or 90%).

For more detailed analysis, here are a few other indispensable options.

What are inodes?
An inode is a data structure on a filesystem that stores all the information about a file except its name and actual data. Every single file or directory requires one inode. It's possible to have a disk with plenty of free space but run out of inodes if you have millions of tiny files. In this situation, you won't be able to create new files. Checking df -i is a key troubleshooting step.

Your Turn: Practice with df

Open your terminal and run the following commands:

1. df -h: Look at the Use% for your root filesystem (/). How full is it?
2. df -hT: What filesystem type is your root partition using?
3. df -i: How many inodes are used versus free on your root filesystem?

This is the exact first step you'd take when an alert says "Disk space is low on server-101."

2. du: Finding the Space Hogs

df told you that your / filesystem is 95% full. Now what? You need to find the specific directory or file that's consuming all the space. This is the job of du, which stands for disk usage. Unlike df, du works by recursively scanning directories and summing up the sizes of the files within them.

Let's see how to use du effectively.

The Ultimate Admin One-Liner

The most common use case for du is to find the largest directories within a specific path. You can do this by combining du with two other commands we've seen: sort and head.

Here’s the pattern:
sudo du -h --max-depth=1 /path/to/check | sort -rh | head -n 10

Let's break that down:

• sudo du -h --max-depth=1 /path/to/check: Get the human-readable sizes of all immediate subdirectories in /path/to/check. We use sudo to ensure we can read all directories.
• |: The pipe operator sends the output of the du command to the input of the next command.
• sort -rh: Sorts the incoming lines. -r reverses the order (largest first), and -h tells it to sort based on human-readable numbers (e.g., it knows 1G is larger than 100M).
• | head -n 10: Takes the sorted list and shows only the top 10 lines.

Your Turn: Practice with du

Let's find the largest directories in your /var directory, a common place for logs and other data to grow.

1. Run sudo du -h --max-depth=1 /var. You'll see the size of each subdirectory (e.g., /var/log, /var/lib).
2. Now, use the one-liner to find the top 5 largest directories:

   sudo du -h --max-depth=1 /var | sort -rh | head -n 5
   

3. Based on the output, you can drill down further. If /var/log was the largest, you could then run sudo du -h --max-depth=1 /var/log | sort -rh | head -n 5 to find the largest log directory, and so on.

This systematic process allows you to quickly pinpoint the source of high disk usage.

3. Why df and du Disagree: An Interview Goldmine

Sooner or later, you will encounter a situation where df reports a disk is 90% full, but when you run du on the root directory, it only accounts for 60% of the space. Where did the "missing" 30% go? This is a classic interview question that tests your depth of knowledge.

The core of the issue lies in how each command gets its information:

• df reads high-level filesystem metadata. It knows the total number of blocks allocated on the disk. It's very fast.
• du walks the directory tree and sums the sizes of all files it can see. It's slower and only reports what's currently visible in the filesystem structure.

The most common reason for a discrepancy is a deleted but still open file.

Here’s the scenario:

1. A web server process is actively writing to a large log file: /var/log/access.log.
2. An administrator, trying to free up space, deletes the file with rm /var/log/access.log.
3. du /var/log no longer sees the file, so its reported size goes down. The administrator thinks the problem is solved.
4. However, the web server process still has the file "open" in the background. The Linux kernel will not free the actual disk blocks until that process closes the file.
5. df reads the filesystem metadata, which knows the blocks are still allocated, so its reported usage does not change.

How to Find and Fix This:

You can find these "ghost" files using the lsof (list open files) command.

sudo lsof | grep '(deleted)'

This command lists all files that are open by a process but have been deleted from the filesystem. The output will show you the command/process name and the process ID (PID) that is holding the file open.

The solution is to gracefully restart the service identified by lsof. When the service restarts, it will close the old file handle, and the kernel will finally release the disk space.

Understanding this distinction between df and du and knowing how to diagnose it with lsof demonstrates a solid grasp of how the Linux filesystem works.

For a deeper dive into this topic, the nixCraft article "Why command df and du reports different output?" (LINK) is a fantastic resource, though not required reading for now.

4. A Modern Alternative: ncdu

While du | sort | head is a powerful and classic technique, it can be cumbersome. Many administrators now use ncdu (NCurses Disk Usage), an interactive tool that makes exploring disk usage much easier.

To try it yourself, first install it:

sudo apt update
sudo apt install ncdu

Then, run it on a directory like /var:

sudo ncdu /var

Wait for the scan to complete, and then use your arrow keys to explore. You can press Enter to go into a directory and Left Arrow to go back up. It's an incredibly efficient way to find space hogs.

Conclusion

You are now equipped with the essential tools and knowledge to handle one of the most common problems in system administration: managing disk space.

Key Takeaways:

• df -h gives a high-level overview of filesystem usage. This is your first step.
• du -h drills down to find the usage of specific directories and files.
• The combo sudo du -h --max-depth=1 <path> | sort -rh | head is your go-to command for finding the largest directories in a given path.
• df and du can disagree, most often due to deleted-but-open files.
• Use sudo lsof | grep '(deleted)' to find the processes holding onto deleted files, then restart those services to free the space.
• ncdu is a powerful, interactive alternative to du for quickly exploring disk usage.

In our next lesson, we will learn how to configure the system's timezone and set up NTP for automatic time synchronization. Accurate time is fundamental to a healthy system. It ensures that the timestamps in your log files (which you might be cleaning up based on du's output!) are correct, that your cron jobs run when expected, and that services across a distributed network can communicate reliably.