Automating Tasks with Cron and At

1. Recurring Tasks: The Power of cron

The most common type of automation is the recurring task. In Linux, this is handled by a daemon called cron. It wakes up every minute, checks a set of configuration files called "crontabs," and executes any tasks scheduled for that minute.

As a DevOps engineer, you will use cron constantly for tasks like:

• Running backup scripts every night.
• Cleaning up temporary files or old logs every week.
• Checking the health of a web service every five minutes.
• Syncing data between servers at off-peak hours.

The crontab Command

Each user on the system can have their own list of scheduled jobs. This list is managed using the crontab command:

• crontab -e: Edit your user's crontab file. This will open the file in your default text editor.
• crontab -l: List the jobs currently in your crontab.
• crontab -r: Remove your entire crontab file. Use this with extreme caution, as it will delete all your scheduled jobs without confirmation.

Understanding cron Syntax

The most challenging part of using cron is its unique syntax for defining a schedule. A crontab entry consists of two parts: the schedule and the command to run.

Let's break down those five fields:

* * * * * command_to_be_executed
┬ ┬ ┬ ┬ ┬
│ │ │ │ │
│ │ │ │ └─── day of week (0 - 6) (Sunday to Saturday; some systems also accept 7 for Sunday)
│ │ │ └──────── month (1 - 12)
│ │ └───────────── day of month (1 - 31)
│ └────────────────── hour (0 - 23)
└─────────────────────── minute (0 - 59)

An asterisk (*) acts as a wildcard, meaning "every." So, * * * * * means "every minute of every hour of every day..."

To get a comprehensive understanding of this syntax and its powerful operators, the following video by Corey Schafer is an excellent, detailed guide.

Syntax Summary and Examples

For convenience, cron also supports special strings for common schedules:

Example: @daily /usr/local/bin/backup.sh is a much more readable way to schedule a daily script.

A Practical DevOps Example: Log Rotation

Let's say you have an application that writes to /var/log/my-app/app.log. This file can grow very large over time. You want to archive the current log and start a fresh one every day at midnight.

1. Create a script: First, you'd create a script, let's call it /opt/scripts/rotate_app_log.sh:

   #!/bin/bash
   
   # Get the current date in YYYY-MM-DD format
   DATE=$(date +%F)
   
   # Move the current log file to an archived name
   mv /var/log/my-app/app.log /var/log/my-app/app.log.$DATE
   
   # Signal the application to reopen its log file (example for a process named 'my-app')
   # This creates a new, empty app.log
   killall -HUP my-app
   

   (Note: You would also need to make this script executable with chmod +x /opt/scripts/rotate_app_log.sh)

2. Schedule it with cron: Open your crontab with crontab -e and add the following line:

   0 0 * * * /opt/scripts/rotate_app_log.sh
   

   Or, using a special string:

   @daily /opt/scripts/rotate_app_log.sh
   

Important: Output and Paths

• Output Redirection: By default, any output from a cron job (both standard output and errors) is emailed to the user who owns the crontab. For automated scripts, you usually want to handle this differently. It's a best practice to redirect output to a log file or discard it.

  • Append all output to a log: >> /var/log/my-cron-job.log 2>&1
  • Discard all output: > /dev/null 2>&1

  Example: 0 0 * * * /opt/scripts/rotate_app_log.sh >> /var/log/my-app/rotation.log 2>&1

• Absolute Paths: cron runs commands in a very minimal environment, which often doesn't have the same PATH variable as your interactive shell. This is a very common reason why scripts that run perfectly in your terminal fail in cron. Always use absolute paths for your scripts and any commands within them.

2. One-Time Tasks with at

What if you only need to run a command once in the future? For example, "reboot this server in 10 minutes" or "run a resource-intensive report at 2 AM tonight." For this, Linux provides the at command.

The at utility is simpler than cron but very powerful for one-off tasks.

First, let's ensure at is installed and its daemon (atd) is running. This is a one-time setup.

Using at

You can schedule a job with at in two main ways:

1. Interactive Mode: Type at followed by a time. The command will then wait for you to enter the commands you want to run. Press Ctrl+D when you're done.

   $ at now + 5 minutes
   warning: commands will be executed using /bin/sh
   at> echo "This is a one-time alert." > /tmp/alert.txt
   at> <Ctrl+D>
   job 1 at Mon Nov 13 14:30:00 2023
   

2. Piping a Command: This is more common for scripting. Use echo to pipe your command directly to at.

   echo "sudo reboot" | at 2:00
   

   This will schedule a reboot for 2 AM. If it's already past 2 AM, it will schedule it for tomorrow.

Time Specifications

One of the best features of at is its flexible, human-readable time format. The same resource we used for installation also provides an excellent overview of the different ways you can specify time.

Managing at Jobs

• atq: Queue of pending jobs. Lists all your scheduled at jobs with their job numbers.
• atrm <job_number>: Remove a scheduled job from the queue.

Conclusion

You've now learned how to control time on your Linux systems! Task scheduling is a fundamental skill that separates manual administration from true automation. By mastering cron for recurring tasks and at for one-time jobs, you can ensure that critical maintenance, backups, and reports happen reliably without any manual intervention.

Key Takeaways:

• Use cron for tasks that need to run on a repeating schedule.
• Manage your recurring tasks with crontab -e (edit) and crontab -l (list).
• The cron syntax has five time/date fields followed by the command. Remember: minute, hour, day-of-month, month, day-of-week.
• Always use absolute paths in your crontab entries to avoid PATH issues.
• Use at for tasks that need to run only once at a specific time in the future.
• at uses a human-friendly time format (e.g., now + 1 hour, 10pm tomorrow).
• Manage one-time tasks with atq (list) and atrm (remove).

In our next lesson, we will cover how to analyze system logs using journalctl and traditional log files in /var/log. This is the perfect follow-up topic, as checking logs is the primary way you'll verify that your scheduled cron jobs have run successfully and troubleshoot them if they haven't.