Input/Output Redirection and Pipes

The Three Standard Streams

Every command you run on the Linux command line operates with at least three standard data streams. Understanding these is the key to controlling the flow of information.

• Standard Input (stdin): Where a command gets its input. By default, this is your keyboard.
• Standard Output (stdout): Where a command sends its successful results. By default, this is your terminal screen.
• Standard Error (stderr): Where a command sends its error messages. By default, this is also your terminal screen.

The following video provides a clear explanation of these three streams in practice.

Redirecting Output: Capturing a Command's Results

What if you don't want a command's output to just appear on the screen? What if you want to save it to a file? This is called output redirection, and it's essential for saving scan results, wordlists, or any other data you generate.

This is done using the > and >> operators.

Let's see how this works.

Key Distinction:

• command > filename.txt: Overwrites the file. Use this when you want a fresh start.
• command >> filename.txt: Appends to the file. Use this when you are building a log or adding to a list.

Redirecting Errors and More

You might have noticed that if a command produces an error, > won't capture it. That's because > only redirects stdout. To control stderr, we need to be more specific.

Linux identifies the standard streams with numbers called file descriptors:

• 0: stdin
• 1: stdout
• 2: stderr

So, > is actually a shortcut for 1>. To redirect errors, you use 2>.

A common and very useful technique in hacking and scripting is to discard unwanted error messages (like "Permission denied" when searching the whole filesystem). You can do this by redirecting stderr to a special "black hole" file called /dev/null.

Example: find / -name "secret.txt" 2> /dev/null
This command searches the entire filesystem for "secret.txt". It will print any results it finds to the screen, but all the "Permission denied" errors will be silently discarded.

Redirecting Input: Feeding a Command from a File

Just as you can redirect output, you can also change a command's input source from the keyboard to a file using the < operator. This is useful for commands that are designed to process data fed into them.

Pipes: Chaining Commands for Maximum Power

This is where your command-line capabilities will take a massive leap. The pipe operator | allows you to take the stdout of one command and use it directly as the stdin for another command. You can chain multiple commands together to create powerful, on-the-fly data processing pipelines.

Let's see this in action.

This technique is incredibly versatile. You can use it to:

• List processes and filter for a specific one: ps aux | grep "apache"
• List files and count how many there are: ls -1 | wc -l
• Find lines in a log file that contain an IP address: cat /var/log/auth.log | grep "192.168.1.101"

Combining All Streams in a Pipeline

By default, the pipe | only passes stdout. What if you want to pass both stdout and stderr to the next command? You need to combine the streams first. The common syntax for this is 2>&1. This tells the shell to "redirect stream 2 (stderr) to the same place as stream 1 (stdout)".

Hands-on Practice

Now it's time to put this all together on your Arch Linux system. Theory is one thing, but muscle memory is built through practice. Open your terminal and work through these challenges.

Challenge 1: Output Redirection

1. Navigate to your home directory (cd ~).
2. Create a new directory for this exercise: mkdir lesson2_practice and enter it: cd lesson2_practice.
3. List the contents of your /etc directory in long format and redirect the output to a file named etc_listing.txt.
   • Hint: ls -l /etc > etc_listing.txt
4. Now, get the current date and append it to the etc_listing.txt file.
   • Hint: date >> etc_listing.txt
5. Verify your work by viewing the file's contents: cat etc_listing.txt. You should see the file list followed by the date.

Challenge 2: Error Redirection

1. Try to view a file that doesn't exist: cat no_such_file. Note the error message on your screen.
2. Now, run the same command but redirect the error message to a file called errors.log. You should see nothing on your screen.
   • Hint: cat no_such_file 2> errors.log
3. Confirm the error was captured: cat errors.log.
4. Try to find all files named hosts on your system, but discard all 'Permission denied' errors.
   • Hint: find / -name "hosts" 2> /dev/null

Challenge 3: Piping and Filtering

1. List all the running processes on your system and use a pipe to filter the output to show only the lines containing the name of your shell (e.g., "bash" or "zsh").
   • Hint: ps aux | grep "bash"
2. Chain three commands: List the contents of /bin, use grep to filter for files containing the letter 'a', and then use wc -l to count how many there are.
   • Hint: ls /bin | grep a | wc -l
3. Combine piping and redirection: Find all the services that systemd is managing, filter for the ones that are "running", and save that list to a file called running_services.txt.
   • Hint: systemctl list-units --type=service | grep "running" > running_services.txt

Conclusion

Excellent work. You've just learned one of the most powerful and flexible features of the Linux command line. These operators are not just conveniences; they are the grammar you use to compose complex operations from simple tools.

Key Takeaways:

• All commands operate with three streams: stdin (0), stdout (1), and stderr (2).
• Redirection changes where these streams point.
  • > redirects stdout, overwriting files.
  • >> redirects stdout, appending to files.
  • < redirects stdin, feeding a file to a command.
  • 2> redirects stderr. Use 2> /dev/null to silence errors.
• The pipe (|) is a connector, sending the stdout of one command to the stdin of the next.
• You can create powerful pipelines by chaining multiple commands to filter, process, and analyze data directly on the command line.

Next Lesson Preview:
In the next lesson, we'll dive into a critical topic for system security: the Linux permission model. You will learn to explain what owner, group, and others mean, and how to interpret permission strings like rwxr-xr--. This is fundamental to understanding how to protect files and how attackers look for misconfigurations to exploit.