Repairing Filesystems with fsck

1. Understanding Filesystem Corruption and the Role of fsck

A filesystem is a complex set of data structures (like inodes, superblocks, and directory entries) that organizes how data is stored and retrieved from a disk. If these structures become inconsistent, the filesystem is considered "corrupted." This can happen for several reasons:

• Improper Shutdown: A sudden power loss or system crash can interrupt writes to the disk, leaving the filesystem in an inconsistent state.
• Hardware Failure: A failing disk drive or controller can write bad data or fail to write data correctly.
• Software Bugs: Errors in the kernel or other low-level software could potentially damage filesystem structures.

When this happens, you might encounter strange behavior: files that disappear, I/O errors when trying to access data, or—most critically—the system failing to boot.

A real-world example of a boot failure. The system has detected filesystem inconsistencies and is instructing the administrator to run fsck manually to fix the problem.

The primary tool for diagnosing and fixing these issues is fsck (File System Consistency Check). It's important to understand what fsck does and, just as importantly, what it doesn't do.

In short, fsck is like a repair service for the filesystem's "address book" (its metadata). It ensures all the pointers and records are correct, but if a page of the book was already scribbled on (corrupted file data), fsck can't restore the original text.

2. How fsck Works: A Wrapper for Specific Tools

One of the most important concepts to grasp is that fsck itself is not one single program. It's a front-end or wrapper that automatically calls the correct tool based on the filesystem type you're trying to check.

• For ext2, ext3, and ext4 filesystems, it calls e2fsck.
• For XFS, it calls xfs_repair (or, more accurately, a small helper that tells you to use xfs_repair).
• For FAT/VFAT, it calls fsck.fat.

This design makes your life easier—you can usually just use the fsck command without needing to memorize the name of every specific checker. However, it's crucial to know that different backend tools have different capabilities and options.

3. The Golden Rule: Unmount Before You Check

This is the most critical rule of using fsck: Never run fsck on a mounted filesystem.

When a filesystem is mounted, the kernel is actively reading from and writing to it. Running fsck at the same time would be like trying to repair a car's engine while it's running. The changes fsck makes could conflict with the kernel's operations, leading to further corruption and catastrophic data loss.

Therefore, the standard workflow is:

1. Identify the device you want to check (e.g., using lsblk or df -h).
2. Unmount it using the umount command.
3. Run fsck.
4. Mount it again using the mount command.

This annotated terminal output clearly shows the correct procedure: first unmount the partition, then run fsck on it.

The only exception to this rule is checking the root filesystem (/), which we will cover in a later section.

4. Practical fsck Usage and Common Options

Now let's look at how to use the command. The basic syntax is:

sudo fsck [options] /dev/device

The article below provides a great walkthrough of the most important options. We'll summarize them here, but use the article as a reference.

Key fsck Options to Know:

• Interactive Repair (Default): sudo fsck /dev/sdb1
  If fsck finds an error, it will stop and ask you what to do. This gives you maximum control but can be tedious if there are many errors.

*An interactive `fsck` session. The tool has found an inconsistency and is prompting the user to choose a course of action.*

• Automatic Repair (-y): sudo fsck -y /dev/sdb1
  This option automatically answers "yes" to every prompt to fix an error. It's fast and convenient but use it with caution, as it will make changes without asking for confirmation.

• Dry Run / Report-Only (-n): sudo fsck -n /dev/sdb1
  This performs a check but does not make any changes. It will only report the errors it finds. This is the safest option and is excellent for assessing the extent of the damage before deciding how to proceed.

• Force Check (-f): sudo fsck -f /dev/sdb1
  By default, fsck will skip checking a filesystem that is marked as "clean" in its superblock. The -f option forces fsck to run a full check even if the filesystem appears to be healthy. This is useful for periodic maintenance checks.

5. Filesystem-Specific Procedures: XFS and ext4

As we learned, fsck is a wrapper. For the two most common Linux filesystems, XFS and ext4, it's good to know the underlying tools and their specific procedures.

Checking XFS Filesystems

You cannot use fsck to repair an XFS filesystem. You must use the dedicated xfs_repair utility.
The standard procedure for checking an XFS filesystem is slightly different because of its robust journaling.

1. Unmount the filesystem.
2. Run xfs_repair -n /dev/device to perform a read-only check. This will report any corruption it finds.
3. If errors are found, run sudo xfs_repair /dev/device to perform the actual repair.
4. In emergencies only: If the journal itself is corrupt and the filesystem won't mount, you may need sudo xfs_repair -L /dev/device. The -L option zeroes the log, which will recover the filesystem but may result in significant data loss from the last few transactions. Use it as a last resort.

The Red Hat documentation (LINK) sections "15.4. Checking an XFS file system with xfs_repair" and "15.5. Repairing an XFS file system with xfs_repair" provide the official, detailed steps.

Checking ext2/3/4 Filesystems

For ext-family filesystems, fsck calls e2fsck. The options we discussed (-y, -n, -f) work as expected. A particularly useful option for e2fsck is -p.

• sudo e2fsck -p /dev/device: The -p option stands for "preen" and performs an automatic, safe repair. It will fix any common, non-destructive issues without prompting but will stop if it encounters a problem that requires administrator intervention. fsck -a is often an alias for e2fsck -p.

6. The Big Challenge: Checking the Root Filesystem

So, if you can't run fsck on a mounted filesystem, how do you check the root filesystem (/), which is always mounted when the system is running? There are two primary methods.

Method 1: Automatic Check on Boot

Linux can be configured to automatically check the root filesystem during the boot process. In the /etc/fstab file, the last column for each entry is a number (pass number) that tells the boot process whether to check the filesystem.

• 0: Do not check.
• 1: Check first (used for the root filesystem).
• 2: Check after the root filesystem (used for all other filesystems).

If the system detects that the root filesystem was not unmounted cleanly, it will automatically run fsck before mounting it read-write.

Method 2: Manual Check via Recovery Mode

If you need to force a manual check of the root filesystem, you cannot do it from your normal running system. You must boot into a special maintenance environment where the root filesystem is mounted read-only or not at all.

This recovery mode approach is the definitive way to safely and manually run fsck on your root partition to fix boot-preventing errors.

Conclusion

You have now reached the end of the "Storage and Filesystem Management" module. You've learned how to partition disks, create filesystems, manage mount points, configure swap, build RAID arrays, and now, how to maintain the health of your filesystems with fsck.

Key Takeaways:

• fsck is the primary Linux utility for checking and repairing filesystem metadata inconsistencies.
• It acts as a wrapper, calling filesystem-specific tools like e2fsck (for ext4) and xfs_repair (for XFS).
• The most important rule is to always unmount a filesystem before checking it.
• Common options include -y (auto-fix), -n (report-only dry run), and -f (force check).
• The root filesystem (/) cannot be unmounted live. It must be checked either automatically at boot or manually by booting into recovery mode.

Preview of the Next Lesson:

In our next module, "Advanced Storage and Networking," we will begin by exploring another powerful storage management technology: Logical Volume Management (LVM). You will learn how to use LVM to create flexible, resizable storage volumes that can span multiple disks, offering a different approach to storage virtualization than the RAID arrays we covered previously.