Managing Background Processes in Linux

1. From Foreground to Background: Taking Back Control

By default, any command you run executes in the foreground. It takes over your terminal, and you can't enter new commands until it finishes. You saw this in the previous lesson when we ran commands like yes or a simple script.

But what if you start a long-running process and then realize you need your terminal back? You don't have to kill it. You can suspend it and move it to the background.

Let's watch a video that demonstrates this entire workflow visually.

Practice Time: The Job Control Loop

Now, let's put what you saw into practice. Open your terminal and follow these steps. We'll use the ping command as our long-running foreground process.

1. Start a foreground process:

   ping google.com
   

   Your terminal is now blocked, showing the ping output.

2. Suspend the process:
   Press Ctrl+Z. You'll see a message like [1]+ Stopped ping google.com, and you'll have your command prompt back. The ping process is now paused.

3. Check your jobs:

   jobs
   

   This lists all jobs associated with your current shell. You'll see your stopped ping command, identified by a job number (e.g., [1]).

4. Resume in the background:
   Let's make ping continue running, but in the background.

   bg %1
   

   (If you only have one job, you can just type bg). You'll see a message like [1]+ ping google.com &, and the ping output will start appearing on your screen again, but you still have your command prompt. You can type other commands while it runs.

5. Bring it back to the foreground:
   Now, let's bring it back to the foreground to interact with it directly.

   fg %1
   

   (Again, fg is sufficient if it's the most recent job). The ping command is now in the foreground again, and your shell is blocked.

6. Terminate the process:
   Press Ctrl+C to send the SIGINT signal and terminate the ping process for good.

This Ctrl+Z -> bg -> fg loop is the fundamental mechanism for managing the state of an already-running process.

2. Starting Processes Directly in the Background

The method above is great for processes you've already started. But if you know from the outset that a command will take a long time, you can launch it directly into the background by adding an ampersand (&) at the end.

Practice Time: Using the Ampersand &

1. Start a background job:
   Let's run a sleep command that will run for 5 minutes (300 seconds) in the background.

   sleep 300 &
   

   The shell will immediately return a job number and a Process ID (PID), and you'll have your prompt back.

2. Verify the job:

   jobs
   

   You'll see your sleep 300 command with the status "Running".

You've now successfully launched a process that runs without blocking your work. To bring it to the foreground, you would use fg %<job_number>, just as before.

3. The Final Challenge: Surviving Disconnection with nohup

You now know how to run tasks in the background. But there's a critical problem: all these jobs are still children of your current shell session. If you close the terminal window or your SSH connection drops, the shell sends a SIGHUP (hangup) signal to all its child processes, causing them to terminate.

In a data center environment, you can't risk your 4-hour backup job dying because of a network glitch. The solution is the nohup command.

nohup stands for "no hangup". When you prepend nohup to a command, it tells that process to ignore the SIGHUP signal. This makes the process immune to terminal closure.

Let's see the classic combination in action: nohup command &.

Practice Time: The Sysadmin's Workhorse

Let's simulate a long-running server task and make sure it survives a "logout".

1. Create a script:
   First, let's create a simple script that logs the current date and time to a file every 5 seconds. This will simulate a monitoring or backup script.

   echo 'while true; do date >> script_output.log; sleep 5; done' > long_runner.sh
   chmod +x long_runner.sh
   

   This creates an executable script named long_runner.sh.

2. Run it with nohup:
   Now, execute the script using nohup and send it to the background with &.

   nohup ./long_runner.sh &
   

   You'll get a message like nohup: ignoring input and appending output to 'nohup.out'. This tells you that since you didn't redirect the script's output, nohup is protecting it by sending it to a file named nohup.out. You'll also get the job and process ID.

3. Verify it's running:
   You can check that the script is running in two ways:

   • jobs: This shows it as a running job in your current shell.
   • tail -f script_output.log: This will show you the live output of your script as the date is written to the log file every 5 seconds. Press Ctrl+C to stop tail.

4. Simulate a logout:
   To prove that nohup works, you need to end your shell session. You can do this by simply typing exit or closing your terminal window.

   Now, open a new terminal window.

5. Confirm persistence:
   In the new terminal, run jobs.

   jobs
   

   The output will be empty! This is because jobs only lists jobs for the current shell. The old shell is gone. But is the process still running? Let's check the system's process list.

   pgrep long_runner.sh
   

   This command should return a PID. Your script is still alive! It has been "disowned" by its original shell and is now running independently.

6. Check the output:
   Finally, look at the log file again.

   tail script_output.log
   

   You will see that the file has continued to grow with new timestamps, proving the script never stopped.

7. Clean up:
   Since the process is no longer a "job" in your new shell, you can't use fg or bg. You must manage it like any other system process, using the skills from the previous lesson. Find its PID and kill it.

   kill $(pgrep long_runner.sh)
   

You've just performed one of the most common and essential tasks for a Linux administrator.

Conclusion

Great job! You've gone beyond simply running and stopping processes. You now have a toolkit for managing the lifecycle of your tasks, allowing you to multitask effectively and run critical jobs with confidence.

Key Takeaways:

• Foreground vs. Background: Foreground processes block the shell; background processes do not.
• Job Control Loop: Use Ctrl+Z to suspend a foreground job, bg to resume it in the background, and fg to bring it back to the foreground.
• & Operator: Append & to a command to start it directly in the background.
• jobs Command: Use jobs to list all background or suspended processes in your current shell.
• nohup Command: Prepend nohup to a command (and usually end with &) to make it immune to terminal closure, which is essential for long-running tasks on remote servers.

In our next lesson, we'll address another critical aspect of administration: "Locate and interpret system logs in /var/log and journalctl to diagnose application or service errors." Now that you know how to run persistent background services, the next logical step is learning how to check their health and troubleshoot them when they have problems by reading their logs.