Rotating Logs With Logrotate in Linux

Log files are essential for system administrators and developers to keep track of what’s happening on a Linux system. Over time, log files can accumulate a significant amount of data, potentially filling up your disk space. To manage and control log files efficiently, Linux provides a powerful utility called Logrotate.

Logrotate is a system utility that manages the automatic rotation and compression of log files on a Linux system. It’s essential for maintaining log file integrity, conserving disk space, and ensuring that log files do not become overwhelming. In this article, we will explore how to use Logrotate effectively, covering its configuration, options, and examples.

Why Rotate Logs?

Before delving into the specifics of Logrotate, let’s understand why log rotation is essential:

1. Disk Space Management: Log files can grow rapidly, consuming valuable disk space. Log rotation ensures that old logs are archived or deleted, preventing them from filling up your storage.
2. Performance: Large log files can impact system performance. By rotating logs, you reduce the size of active log files, making it faster and easier to search through them.
3. Security: Maintaining logs for an extended period can pose security risks, as they may contain sensitive information. Regular rotation and deletion help protect sensitive data.
4. Log File Maintenance: Logrotate automates the process of log file management, making it easier to maintain logs without manual intervention.

Basic Logrotate Configuration

Logrotate is highly configurable, and its settings are defined in configuration files located in the /etc/logrotate.d/ directory. The primary configuration file is /etc/logrotate.conf, but most custom configurations are stored in separate files within the /etc/logrotate.d/ directory.

Let’s take a look at the basic structure of a Logrotate configuration file:

/path/to/log/file {
    rotate N
    weekly
    missingok
    notifempty
    compress
    delaycompress
    create 0640 user group
}

Here’s a breakdown of these directives:

• /path/to/log/file: Specifies the log file that needs rotation.
• rotate N: Retains the last N log files and removes older ones. For example, rotate 5 keeps the last five log files.
• weekly: Sets the rotation schedule. Other options include daily, monthly, and yearly.
• missingok: Ignores the error if the log file is missing.
• notifempty: Skips rotation if the log file is empty.
• compress: Compresses rotated log files.
• delaycompress: Delays compression until the next rotation cycle.
• create 0640 user group: Creates a new log file with the specified permissions, user, and group if it does not exist.

Logrotate Options

Logrotate provides various options that you can use to customize the behavior of log rotation:

• size: Rotates the log file when it reaches a specified size. For example, size 10M rotates the log when it reaches 10 megabytes.
• copytruncate: Copies the current log file content to a new file and truncates the original file. Useful for applications that do not support log file rotation.
• postrotate and prerotate: Executes custom shell commands before or after log rotation. Useful for tasks such as restarting services after rotation.
• dateext: Appends the date to rotated log files, ensuring unique file names.

Logrotate Examples

Let’s explore a few practical examples of Logrotate configurations:

1. Rotating Apache Logs

/var/log/apache2/*.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    create 0640 root adm
    sharedscripts
    postrotate
        /etc/init.d/apache2 reload > /dev/null
    endscript
}

This configuration rotates Apache logs weekly, keeping the last four rotated files. It also compresses them and reloads the Apache service after rotation.

2. Rotating Application-specific Logs

/var/log/myapp/*.log {
    daily
    rotate 7
    size 50M
    missingok
    notifempty
    delaycompress
    create 0644 myapp myapp
}

Here, we rotate logs generated by a hypothetical application daily, retaining the last seven logs. Rotation also occurs when the log file reaches 50 megabytes.

3. Rotating System Logs

/var/log/syslog /var/log/auth.log {
    weekly
    rotate 4
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

This configuration rotates system logs, including syslog and auth.log, weekly, and keeps the last four logs. It also uses rsyslog-rotate to signal the rsyslog daemon after rotation.

Testing Logrotate Configuration

Before relying on your Logrotate configuration, it’s a good practice to test it. You can do this using the -d or --debug option:

sudo logrotate -d /etc/logrotate.conf

This command performs a dry run, displaying what Logrotate would do without actually rotating the logs.

Manually Triggering Log Rotation

By default, Logrotate is scheduled to run daily via a cron job. However, you can manually trigger log rotation using the following command:

sudo logrotate -f /etc/logrotate.conf

The -f option forces Logrotate to perform log rotation immediately.

Advanced Logrotate Features

In addition to the basics of log rotation, Logrotate offers some advanced features and techniques that can help you fine-tune your log file management:

1. Multiple Log File Patterns

You can specify multiple log file patterns in a single Logrotate configuration file. For instance, if you want to rotate logs from different directories, you can do so like this:

/var/log/app1/*.log /var/log/app2/*.log {
    # Configuration options here
}

This allows you to manage logs from various sources in one central configuration.

2. Using Wildcards

Logrotate supports wildcard characters to match log files more flexibly. For example, you can use * to match any character sequence and ? to match any single character. Here’s an example:

/var/log/app-*/access.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
}

This configuration would match log files like /var/log/app-1/access.log and /var/log/app-2/access.log, rotating them according to the specified settings.

3. Custom Scripts

You can integrate custom scripts into Logrotate configurations using postrotate and prerotate directives. For instance, you may want to email log files to yourself after rotation or perform other custom actions:

/var/log/custom.log {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    postrotate
        /usr/local/bin/custom-log-action.sh
    endscript
}

In this example, after the log file is rotated, the custom-log-action.sh script is executed.

4. Creating Custom Configuration Files

While the /etc/logrotate.conf file contains global settings, it’s often a good practice to create separate configuration files for each application or service in the /etc/logrotate.d/ directory. This modular approach makes it easier to manage and organize log rotation rules.

For instance, if you have a web server, you might create a file named /etc/logrotate.d/httpd for its log rotation settings.

5. Logging and Debugging

You can specify a debug option in your configuration to enable debug mode. This will log detailed information about Logrotate’s actions, which can be useful for troubleshooting:

/var/log/debug.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    debug
}

This configuration will log debugging information to the syslog.

Monitoring Logrotate

Logrotate runs automatically as a cron job, but it’s crucial to monitor its activity to ensure that log rotation occurs as expected. You can check the logrotate status and history in the /var/lib/logrotate/status file.

Additionally, you can set up email notifications for log rotation activities. Configure the mail directive in your Logrotate configuration to receive email notifications when log files are rotated.

Conclusion

Logrotate is a powerful tool for managing log files on a Linux system efficiently. By carefully configuring log rotation rules and taking advantage of its advanced features, you can ensure that your log files remain organized, consume an appropriate amount of disk space, and aid in system maintenance and troubleshooting.

Regularly reviewing and optimizing your Logrotate configurations is essential to adapt to changing requirements and maintain a healthy log management strategy. With the knowledge of Logrotate, you can keep your system’s log files in check and improve overall system stability and security.

For more blogs, please click on Linux.