Inspecting Storage Health with `smartctl`

1. Understanding S.M.A.R.T. and smartctl

Before we jump into the practical side, it's important to understand what we're working with. Modern storage devices (both HDDs and SSDs) include a built-in monitoring system called S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology). This system constantly tracks various metrics about the drive's health and performance.

Our window into this system on Linux is a utility called smartctl. To get a formal introduction to these concepts, please read the first few sections of the following article.

The smartctl tool is part of the smartmontools package. Let's start by getting it installed on your system. This is a quick practical step you'll perform on almost any new server you manage.

Open your terminal and run:

sudo apt update
sudo apt install smartmontools

To verify the installation, you can run smartctl --version.

2. Performing Your First Health Check

With smartctl installed, you can immediately check the health of your drives. The most direct way to do this is with the -H or --health option, which gives a simple "PASSED" or "FAILED" assessment.

First, you need to identify your storage device's name. You can use lsblk to list your block devices. It will likely be something like /dev/sda, /dev/nvme0n1, etc. For this lesson, we'll assume your drive is /dev/sda. Remember to replace this with your actual device name.

# Run this command, replacing /dev/sda with your drive name
sudo smartctl -H /dev/sda

A PASSED result is good, but it doesn't tell the whole story. A drive can pass this basic check while still showing early warning signs of trouble. For a complete picture, we use the -a or --all option.

The following video demonstrates a real-world scenario of a failing drive and shows the initial smartctl -a command.

The output of smartctl -a is dense, but it's typically broken into three main sections:

1. Device Information: Model number, serial number, firmware version, etc.
2. Overall Health Assessment: The same "PASSED" or "FAILED" status from the -H flag.
3. S.M.A.R.T. Attributes Table: The heart of the report, containing detailed metrics.

3. Interpreting Key S.M.A.R.T. Attributes

This is the most crucial skill. Knowing which attributes to watch for is what separates a novice from an experienced admin. The values can be cryptic, and their exact meaning can vary by manufacturer. However, a few attributes are universally critical.

Let's look at a few of the most important ones. The table below lists attributes that are frequently monitored and are excellent to know for a job interview.

The following video segment explains the challenge of interpreting these attributes and focuses on the Current_Pending_Sector_Count, which is one of the most significant predictors of failure.

4. Actively Testing Your Drives

Waiting for S.M.A.R.T. attributes to change is a passive approach. You can also actively command the drive to test itself. These tests are non-destructive, meaning they won't harm your data and can be run on a live system.

There are two main types of tests:

• Short: Takes a few minutes. Performs a quick check of the drive's electrical and mechanical systems.
• Long (or Extended): Can take several hours. Scans the entire disk surface for defects. It's a much more thorough check.

You can initiate these tests with the -t or --test option.

# Start a short self-test
sudo smartctl -t short /dev/sda




# Start a long self-test
sudo smartctl -t long /dev/sda

After starting a test, the command will tell you how long it's estimated to take. The results aren't shown immediately. You can check the progress and see the final results by viewing the full report (-a) or by specifically looking at the self-test log with -l selftest.

This final video clip will show you exactly how to run these tests and check the results.

A common practice for sysadmins is to schedule a long self-test to run automatically once a week or once a month (using cron, which you've learned about) during off-peak hours.

Conclusion

You now have the knowledge and practical skills to assess the health of the most failure-prone components in a server: the storage drives. By using smartctl, you can move from a reactive to a proactive maintenance strategy, identifying potential problems before they result in catastrophic data loss and downtime.

Key Takeaways:

• S.M.A.R.T. is a built-in drive monitoring system, and smartctl is your command-line tool to access it.
• A quick health summary is available with smartctl -H, but a full report with smartctl -a is necessary for a real diagnosis.
• Focus on key attributes: Reallocated_Sector_Ct (ID 5), Current_Pending_Sector_Ct (ID 197), and Power_On_Hours (ID 9) are critical indicators of drive health and age.
• UDMA_CRC_Error_Count (ID 199) often points to cabling issues, not a failing drive.
• You can and should run non-destructive self-tests (-t short, -t long) to actively check for problems.

In our next lesson, we will continue with essential maintenance tasks by learning how to apply security updates and kernel patches, and manage system reboots for kernel upgrades. Keeping the core of the operating system updated is just as important as ensuring the hardware is healthy.