Hello and welcome to our next lesson!
In our last session, you learned the essential commands for managing the file system: mkdir, cp, mv, and rm. You now know how to create, organize, and delete files and directories, which are the "containers" of your work.
Today, we're going to look inside those containers. In your DevOps role, you'll constantly need to read log files, check configurations, and find specific pieces of information within text. This lesson focuses on the fundamental commands for viewing and searching the contents of files.
By the end of this 60-minute lesson, you will be able to view and manipulate file contents using cat, less, head, tail, and grep.
Let's get started. First, open your terminal and navigate to the practice directory we created in the last lesson.
cd /tmp/tutorial
Next, let's create a sample log file to work with. This file simulates the kind of output you might see from an application. Run these commands one by one:
echo "INFO: Application starting up." > app.log
echo "DEBUG: Connecting to database on host: db.internal." >> app.log
echo "INFO: Database connection successful." >> app.log
echo "WARNING: High memory usage detected." >> app.log
echo "ERROR: Failed to process request #123. Reason: Timeout." >> app.log
echo "INFO: Shutting down." >> app.log
echo "Error: Another failure occurred." >> app.log
(Remember, > creates a file and overwrites existing content, while >> appends to the end of the file.)
1. Viewing Entire Files: cat and less
The most direct way to view a file is to print its entire contents to the screen. For this, we have two primary tools: cat and less.
cat(short for concatenate) reads a file and displays its full content in the terminal. It's fast and simple, making it perfect for small files.
Try it now with the app.log file you just created:
cat app.log
You'll see the entire, short log file printed to your screen. But what if the file had thousands of lines? cat would dump all of them, and you'd only see the last few that fit on your screen. This is where less comes in.
lessis a pager. It displays a file's content one "page" (one screen) at a time, allowing you to scroll up and down. This makes it ideal for viewing large files like system logs or long configuration files.
The following reading explains the use of cat and introduces the more powerful less command for navigating larger files.
This guide from the University of Texas introduces the basic commands for viewing files. It clearly explains the limitation of cat and how less solves it by allowing you to page through content.
Please read the first section titled 'cat, more or less'. Focus on understanding the difference between cat and less, and familiarize yourself with the basic navigation commands for less (like 'q' to quit, spacebar to advance, and the arrow keys).
After the reading, try opening a longer system file with less to see its paging capability in action. For example, try less /var/log/syslog (you may need to use sudo less /var/log/syslog depending on permissions). Press q to exit less when you're done.
2. Viewing Parts of Files: head and tail
Often, you don't need to see the entire file. You might only need the first few lines to check a file's header, or the last few lines to see the most recent log entries. For this, we use head and tail.
This simple visual analogy can help you remember their functions:
head: Shows the beginning of a file. By default, it shows the first 10 lines.tail: Shows the end of a file. By default, it shows the last 10 lines.
Both commands accept the -n option to specify a different number of lines. For example, head -n 5 app.log shows the first five lines.
A DevOps Superpower: tail -f
The tail command has a special option, -f (for "follow"), that is invaluable for DevOps work. It watches a file and prints new lines as they are added in real-time. This is perfect for monitoring live application logs.
If you were to run tail -f app.log in one terminal window and then run echo "NEW LOG ENTRY" >> app.log in another, you would see the new line appear instantly in the first window.
The following reading covers head and tail in more detail, including a useful trick for viewing lines from the middle of a file.
Let's dive into the specifics of head and tail. This resource covers their basic usage and introduces an advanced technique using tail with a + to start viewing from a specific line number.
Read the section 'head and tail'. Pay attention to how the -n option works and how you can use tail -n +<number> to view a file starting from a specific line.
Test your understanding!
Using the app.log file we created, which has 7 lines, how would you display only the 4th line, which reads "WARNING: High memory usage detected."? (Hint: You'll need to combine head and tail with a pipe |.)
Show answer
There are a couple of ways to do this. Here's a common one:
head -n 4 app.log | tail -n 1
head -n 4 app.log: This command takes the first 4 lines of the file.|: The pipe sends the output of theheadcommand as the input to thetailcommand.tail -n 1: This command takes the last 1 line of the input it received.
The result is the 4th line of the original file.
3. Searching Inside Files: grep
Viewing files is useful, but often you need to find something specific. The grep command (Global Regular Expression Print) is the primary tool for this. It searches files for lines containing a matching pattern and prints them.
For example, to find all the informational messages in our log file, you could run:
grep "INFO" app.log
grep is case-sensitive by default. Notice what happens if you search for "error" in lowercase:
grep "error" app.log
It only finds the last line, ignoring the "ERROR" in uppercase. This is where options become powerful.
Common grep options for DevOps:
-i: Ignore case. (grep -i "error" app.logwould find both "ERROR" and "Error").-n: Show line numbers.-v: Invert match (show lines that do not match the pattern).-c: Print a count of matching lines.-ror-R: Search recursively through a directory. (e.g.,grep -r "database_password" /etc/to find a setting in any config file within that directory).
The next reading introduces grep and some of its most useful options.
This tutorial provides a clear introduction to grep, explaining how to search for patterns and use essential options to refine your search.
Read the section '2.5 Searching the contents of a file'. Start from 'grep' and continue to the end of the section. Pay close attention to the examples for the -i, -n, -v, and -c options.
Test your understanding!
Using the app.log file, which single command would you use to count how many lines contain the word "error", regardless of its case (i.e., matching both "ERROR" and "Error")?
Show answer
The correct command is: grep -ic "error" app.log
grep: The search command.-i: Makes the search case-insensitive.-c: Tellsgrepto output only the count of matching lines, not the lines themselves."error": The pattern to search for.app.log: The file to search in.
This command should output 2.
Conclusion
Excellent work! You now have a powerful set of tools for inspecting file contents, which is a daily task in any DevOps role. Whether you're debugging an issue by reading logs, verifying a configuration, or searching for a piece of code, these commands are your go-to utilities.
Key Takeaways:
cat: Best for a quick view of small files.less: Essential for browsing large files page by page.head/tail: Perfect for checking the start or end of a file.tail -f: Your tool for monitoring files like logs in real-time.grep: The ultimate tool for searching for specific text within files, with powerful options to customize the search.
In our next lesson, we will build on this by learning how to manage file permissions. Now that you know how to create, manage, and read files, the next logical step is to control who has the right to perform these actions. We'll explore the chmod, chown, and umask commands to secure your files and directories.