What Is Linux Log Rotation?

Linux log rotation is an automated system maintenance process that manages log files by compressing, archiving, and deleting old entries to prevent disk space exhaustion. The primary tools—logrotate, rsyslog, and journalctl—work together to ensure your system maintains optimal performance while preserving critical diagnostic information.

Quick Start Command:

sudo logrotate -f /etc/logrotate.conf

This single command forces immediate rotation of all configured logs, providing instant relief for systems experiencing disk space issues.

---

Table of Contents

• How Does Linux Log Rotation Work?
• Why Is Log Rotation Critical for System Health?
• What Tools Handle Log Rotation in Linux?
• How to Configure Logrotate for System Logs
• How to Set Up Advanced Log Rotation Policies
• How to Monitor Log Rotation Status
• How to Handle Application-Specific Log Rotation
• What Are Best Practices for Log Retention?
• Troubleshooting Common Log Rotation Issues
• Frequently Asked Questions
• Related Resources and Advanced Topics

---

How Does Linux Log Rotation Work?

Understanding the log rotation mechanism is essential for effective system administration. Consequently, log rotation operates through a scheduled process that monitors log file sizes and ages, then executes predefined actions when thresholds are met.

The Log Rotation Lifecycle

The rotation process follows a systematic approach:

1. Detection Phase: The system identifies logs requiring rotation based on size, age, or schedule
2. Preparation Stage: Current log files are closed by running services
3. Rotation Execution: Files are renamed with timestamps or numerical suffixes
4. Compression Step: Older logs are compressed using gzip or bzip2
5. Cleanup Process: Ancient logs exceeding retention policies are permanently deleted

Moreover, each phase includes safety checks to prevent data loss during the rotation process.

Key Components in Log Management Systems

Component	Purpose	Configuration Location
logrotate	Primary rotation utility	/etc/logrotate.conf
rsyslog	System logging daemon	/etc/rsyslog.conf
journald	systemd journal manager	/etc/systemd/journald.conf
Cron jobs	Scheduling mechanism	/etc/cron.daily/logrotate

Why Is Log Rotation Critical for System Health?

Implementing proper log file management prevents several catastrophic scenarios. First and foremost, unmanaged logs can consume entire disk partitions, causing system failures and service interruptions.

Disk Space Prevention

Without rotation, log files grow indefinitely. For instance, a busy web server can generate gigabytes of access logs daily. Therefore, rotation ensures:

• Partition protection: Prevents /var filesystem from filling completely
• Performance maintenance: Reduces I/O overhead from massive log files
• Service stability: Avoids crashes caused by write failures

Compliance and Audit Requirements

Additionally, many organizations must retain logs for specific periods to meet regulatory requirements. Log rotation helps balance:

• Security incident investigation needs
• Storage cost optimization
• Data retention policies compliance

What Tools Handle Log Rotation in Linux?

Three primary utilities manage log rotation across modern Linux distributions. Each tool serves specific purposes and integrates seamlessly with system services.

Understanding Logrotate Configuration

The logrotate utility serves as the cornerstone of log management. It reads configuration files and executes rotation policies accordingly.

Basic Logrotate Syntax:

# Check logrotate configuration syntax
sudo logrotate -d /etc/logrotate.conf

# View current rotation status
sudo cat /var/lib/logrotate/status

# Force rotation for specific log
sudo logrotate -f /etc/logrotate.d/apache2

How to Configure Logrotate for System Logs

Create custom rotation policies by understanding the configuration directives. Here’s a comprehensive example:

# Create custom configuration
sudo nano /etc/logrotate.d/myapp

# Add the following configuration:
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp
    sharedscripts
    postrotate
        systemctl reload myapp > /dev/null 2>&1 || true
    endscript
}

Configuration Directive Breakdown:

Directive	Function	Example Value
daily	Rotation frequency	daily, weekly, monthly
rotate 14	Number of old logs to keep	7, 30, 52
compress	Enable gzip compression	compress / nocompress
delaycompress	Skip compression for most recent	delaycompress
missingok	Ignore missing log files	missingok
notifempty	Skip rotation if log is empty	notifempty
create	Set permissions for new log	0640 user group

Managing Systemd Journal Logs

Furthermore, journald manages systemd service logs differently than traditional text files. Therefore, specific commands control journal rotation:

# Check current journal disk usage
sudo journalctl --disk-usage

# Configure journal size limits
sudo nano /etc/systemd/journald.conf

# Add or modify these settings:
[Journal]
SystemMaxUse=500M
SystemKeepFree=1G
SystemMaxFileSize=100M
MaxRetentionSec=1month

# Apply changes
sudo systemctl restart systemd-journald

# Manually clean old journal entries
sudo journalctl --vacuum-time=7d
sudo journalctl --vacuum-size=500M

Configuring Rsyslog for Automatic Rotation

Rsyslog handles real-time log collection and forwarding. However, it relies on logrotate for file management:

# Configure rsyslog output
sudo nano /etc/rsyslog.d/50-default.conf

# Example remote logging configuration
*.* @@remote-log-server:514

# Local file logging with rotation preparation
$outchannel log_rotation,/var/log/myapp.log,52428800,/usr/bin/logrotate /etc/logrotate.d/myapp
*.* :omfile:$log_rotation

# Test configuration
sudo rsyslogd -N1

# Restart service
sudo systemctl restart rsyslog

How to Set Up Advanced Log Rotation Policies

Advanced configurations address complex enterprise requirements. Subsequently, these examples demonstrate sophisticated rotation strategies.

Time-Based Rotation Strategy

Implement rotation based on specific time intervals:

# Hourly log rotation for high-volume applications
/var/log/highvolume/*.log {
    hourly
    rotate 168
    compress
    dateext
    dateformat -%Y%m%d-%H
    missingok
    notifempty
}

Size-Based Rotation Implementation

Alternatively, rotate logs when they reach specific sizes:

# Size-based rotation example
/var/log/database/*.log {
    size 100M
    rotate 10
    compress
    create 0644 postgres postgres
    postrotate
        /usr/bin/killall -HUP postgres 2>/dev/null || true
    endscript
}

Multi-Log Rotation Configuration

Manage multiple log files with shared policies:

# Rotate multiple logs with single configuration
/var/log/app1/*.log /var/log/app2/*.log /var/log/app3/*.log {
    weekly
    rotate 4
    compress
    sharedscripts
    postrotate
        systemctl reload app1 app2 app3 > /dev/null 2>&1
    endscript
}

How to Monitor Log Rotation Status

Monitoring ensures your rotation policies execute correctly. Therefore, implement these verification techniques regularly.

Checking Rotation Execution

# View logrotate status file
sudo cat /var/lib/logrotate/status

# Check last rotation dates
ls -lht /var/log/ | head -20

# Verify compressed logs exist
find /var/log -name "*.gz" -mtime -7

# Monitor logrotate execution via journald
sudo journalctl -u logrotate -n 50

Testing Rotation Policies

Test configurations before deploying to production:

# Debug mode (doesn't actually rotate)
sudo logrotate -d /etc/logrotate.d/myapp

# Verbose mode (shows detailed execution)
sudo logrotate -v /etc/logrotate.d/myapp

# Force rotation regardless of schedule
sudo logrotate -f /etc/logrotate.d/myapp

# Test specific configuration file
sudo logrotate --state /tmp/logrotate-test.state -f /etc/logrotate.d/myapp

Automated Rotation Monitoring Script

Create a monitoring script for proactive alerting:

#!/bin/bash
# Log rotation monitoring script
# Save as: /usr/local/bin/check-log-rotation.sh

LOGROTATE_STATUS="/var/lib/logrotate/status"
ALERT_THRESHOLD=86400  # 24 hours in seconds
CURRENT_TIME=$(date +%s)

while IFS= read -r line; do
    if [[ $line =~ ^\" ]]; then
        logfile=$(echo $line | cut -d'"' -f2)
        timestamp=$(echo $line | awk '{print $2,$3}')
        log_epoch=$(date -d "$timestamp" +%s 2>/dev/null)
        
        if [ $? -eq 0 ]; then
            time_diff=$((CURRENT_TIME - log_epoch))
            
            if [ $time_diff -gt $ALERT_THRESHOLD ]; then
                echo "WARNING: $logfile has not rotated in $((time_diff / 3600)) hours"
            fi
        fi
    fi
done < "$LOGROTATE_STATUS"

# Check disk usage
DISK_USAGE=$(df -h /var/log | awk 'NR==2 {print $5}' | sed 's/%//')
if [ $DISK_USAGE -gt 80 ]; then
    echo "CRITICAL: /var/log partition is ${DISK_USAGE}% full"
fi

Make the script executable and add to cron:

sudo chmod +x /usr/local/bin/check-log-rotation.sh

# Add to crontab for hourly checks
sudo crontab -e
# Add: 0 * * * * /usr/local/bin/check-log-rotation.sh | mail -s "Log Rotation Status" admin@example.com

How to Handle Application-Specific Log Rotation

Different applications require tailored rotation strategies. Thus, understanding application logging patterns optimizes your configuration.

Rotating Web Server Logs

Apache HTTP Server Configuration:

# Apache logrotate configuration
sudo nano /etc/logrotate.d/apache2

/var/log/apache2/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        if invoke-rc.d apache2 status > /dev/null 2>&1; then \
            invoke-rc.d apache2 reload > /dev/null 2>&1; \
        fi
    endscript
}

Nginx Configuration:

# Nginx logrotate configuration
sudo nano /etc/logrotate.d/nginx

/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        [ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
    endscript
}

Rotating Database Logs

MySQL/MariaDB Configuration:

# MySQL log rotation
sudo nano /etc/logrotate.d/mysql

/var/log/mysql/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql adm
    sharedscripts
    postrotate
        test -x /usr/bin/mysqladmin && \
        /usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf flush-logs
    endscript
}

PostgreSQL Configuration:

# PostgreSQL log rotation
sudo nano /etc/logrotate.d/postgresql

/var/log/postgresql/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 postgres postgres
    sharedscripts
    postrotate
        /usr/bin/pg_ctl reload -D /var/lib/postgresql/14/main
    endscript
}

What Are Best Practices for Log Retention?

Implementing effective retention policies balances storage costs with operational needs. Consequently, consider these industry-standard approaches.

Retention Policy Matrix

Log Type	Retention Period	Compression	Archive Location
System logs	90 days	After 7 days	Local disk
Security logs	1 year	After 30 days	Remote storage
Application logs	30 days	After 3 days	Local disk
Access logs	60 days	After 1 day	Local/Remote
Debug logs	7 days	Immediate	Local disk

Implementing Tiered Storage

Move older logs to cheaper storage automatically:

# Create archive script
sudo nano /usr/local/bin/archive-old-logs.sh

#!/bin/bash
# Archive logs older than 30 days to remote storage

ARCHIVE_DIR="/mnt/archive/logs"
LOG_DIR="/var/log"
DAYS_OLD=30

# Find and move old compressed logs
find $LOG_DIR -name "*.gz" -mtime +$DAYS_OLD -type f | while read file; do
    relative_path="${file#$LOG_DIR/}"
    target_dir="$ARCHIVE_DIR/$(dirname $relative_path)"
    
    mkdir -p "$target_dir"
    mv "$file" "$target_dir/" && echo "Archived: $file"
done

# Sync to remote storage
rsync -avz --remove-source-files $ARCHIVE_DIR/ backup@remote:/backup/logs/

Troubleshooting Common Log Rotation Issues

Even well-configured systems encounter rotation problems. Therefore, use these diagnostic techniques to identify and resolve issues quickly.

Issue: Logrotate Not Executing

Symptoms:

• Logs continue growing without rotation
• No compressed log files appearing
• Status file shows old dates

Diagnostic Steps:

# Verify logrotate is in cron
ls -l /etc/cron.daily/logrotate

# Check cron execution
sudo grep logrotate /var/log/syslog

# Test manual execution
sudo /etc/cron.daily/logrotate

# Check for errors
sudo logrotate -v /etc/logrotate.conf 2>&1 | grep -i error

Solutions:

# Fix permissions on logrotate binary
sudo chmod 755 /usr/sbin/logrotate

# Repair status file
sudo rm /var/lib/logrotate/status
sudo touch /var/lib/logrotate/status
sudo chmod 644 /var/lib/logrotate/status

# Re-enable cron job if disabled
sudo systemctl status cron
sudo systemctl restart cron

Issue: Postrotate Scripts Failing

Symptoms:

• Rotation completes but services don’t reload
• Applications continue writing to rotated files
• Empty new log files

Diagnostic Commands:

# Enable debug output
sudo logrotate -d -v /etc/logrotate.d/myapp 2>&1 | grep -A5 postrotate

# Check script syntax
bash -n /etc/logrotate.d/myapp

# Test postrotate command independently
sudo systemctl reload myapp
echo $?  # Should return 0 for success

Solutions:

# Add error handling to postrotate
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    sharedscripts
    postrotate
        if systemctl is-active --quiet myapp; then
            systemctl reload myapp >> /var/log/logrotate-myapp.log 2>&1
        fi
    endscript
}

# Create dedicated log for rotation errors
sudo touch /var/log/logrotate-errors.log
sudo chmod 640 /var/log/logrotate-errors.log

Issue: Insufficient Disk Space During Rotation

Symptoms:

• Rotation starts but fails mid-process
• Temporary files accumulate
• System shows out of space errors

Diagnostic Commands:

# Check disk space
df -h /var/log

# Identify large log files
du -sh /var/log/* | sort -hr | head -10

# Check for temporary rotation files
find /var/log -name "*.log.*" -type f

# Monitor real-time disk usage during rotation
watch -n 1 'df -h /var/log'

Emergency Recovery:

# Immediate space recovery
sudo find /var/log -name "*.log.*" -mtime +7 -delete
sudo journalctl --vacuum-size=100M

# Compress current logs without rotation
sudo find /var/log -name "*.log" -size +100M -exec gzip {} \;

# Temporarily disable verbose logging
sudo systemctl set-log-level notice

Issue: Compression Failures

Symptoms:

• Rotated logs exist but aren’t compressed
• Partial .gz files present
• Compression errors in logs

Diagnostic Steps:

# Check gzip availability and version
which gzip
gzip --version

# Test compression manually
sudo gzip -t /var/log/app.log.1.gz
echo $?  # 0 = success, non-zero = corruption

# Verify compression configuration
sudo grep -i compress /etc/logrotate.d/*

Solutions:

# Reinstall compression utilities
sudo apt install --reinstall gzip bzip2  # Debian/Ubuntu
sudo yum reinstall gzip bzip2            # RHEL/CentOS

# Switch to alternative compression
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    compresscmd /usr/bin/bzip2
    compressext .bz2
    delaycompress
}

# Repair corrupted compressed logs
find /var/log -name "*.gz" -exec gzip -t {} \; -print 2>&1 | grep -i failed | awk '{print $2}' | xargs sudo rm -f

Issue: Permission Denied Errors

Symptoms:

• Rotation fails with permission errors
• Cannot create new log files
• Services fail to write after rotation

Diagnostic Commands:

# Check log directory permissions
ls -la /var/log/myapp/

# Verify logrotate can access files
sudo -u root test -r /var/log/myapp/app.log && echo "Readable"
sudo -u root test -w /var/log/myapp/ && echo "Writable"

# Check SELinux/AppArmor context
ls -Z /var/log/myapp/

Solutions:

# Fix ownership and permissions
sudo chown -R myapp:myapp /var/log/myapp
sudo chmod 755 /var/log/myapp
sudo chmod 644 /var/log/myapp/*.log

# Ensure create directive has correct permissions
/var/log/myapp/*.log {
    daily
    rotate 14
    create 0644 myapp myapp
    su myapp myapp
}

# Fix SELinux context (if applicable)
sudo restorecon -R /var/log/myapp

Frequently Asked Questions

How often should log rotation run?

The rotation frequency depends on log volume and disk capacity. Generally, system logs rotate daily, high-volume application logs may rotate hourly, and low-traffic logs can rotate weekly. Moreover, monitor your disk usage patterns for the first month to optimize the schedule.

What happens to logs during rotation?

During rotation, the current log file is renamed (e.g., app.log becomes app.log.1), and a fresh app.log file is created. Subsequently, the application receives a signal to start writing to the new file. Older numbered logs shift down (.1 becomes .2, etc.), and the oldest log exceeding your retention policy is deleted or archived.

Can I rotate logs without stopping services?

Yes, absolutely. Modern applications support log reopening via signals like SIGHUP or SIGUSR1. The postrotate script in logrotate configuration sends these signals, allowing seamless rotation without service interruption. However, some legacy applications may require brief restarts.

How much disk space does compression save?

Text log files typically compress to 5-10% of their original size using gzip. For instance, a 1GB log file usually compresses to 50-100MB. Furthermore, compression ratios improve with repetitive content like HTTP access logs.

What’s the difference between logrotate and journalctl vacuum?

Logrotate manages traditional text-based log files in /var/log, while journalctl vacuum manages binary systemd journal files. Therefore, both tools are necessary on modern systems—logrotate for application logs and journalctl for system service logs.

Should I rotate logs to remote storage?

For critical systems, yes. Implement a two-tier strategy: keep recent logs locally for quick access and archive older logs to remote storage for long-term retention and disaster recovery. Additionally, remote storage protects logs from local system compromises.

How do I recover from deleted logs?

Prevention is key—implement backups before deletion. However, if logs are accidentally deleted, immediately check:

1. Remote syslog servers if configured
2. System journal: sudo journalctl --since "2 days ago"
3. Application-specific log buffers
4. File recovery tools like extundelete (unmount partition first)

Can logrotate handle logs on mounted filesystems?

Yes, logrotate works with any mounted filesystem. Specify the full path in your configuration file. Additionally, ensure the missingok directive is set to prevent errors if the mount point becomes unavailable.

What are copytruncate and its use cases?

The copytruncate directive copies the log file’s contents and truncates the original rather than moving it. Use this for applications that cannot reopen log files or lack signal handling. However, note that a small window exists where log entries might be lost during the copy operation.

/var/log/legacyapp/*.log {
    daily
    rotate 7
    copytruncate
    compress
    notifempty
}

How do I test rotation without affecting production?

Use debug mode with a test state file:

# Create test configuration
sudo cp /etc/logrotate.d/myapp /tmp/test-rotation.conf

# Test without making changes
sudo logrotate -d -v /tmp/test-rotation.conf

# Test with isolated state file
sudo logrotate --state /tmp/test-state -f /tmp/test-rotation.conf

This approach allows safe testing without impacting your production rotation schedule.

Related Resources and Advanced Topics

To deepen your Linux system administration expertise, explore these related guides on LinuxTips.pro:

Previous LinuxTips.pro Articles:

• Process Management: ps, top, htop, and kill – Monitor processes that generate logs
• System Services with systemd – Understand how services interact with log rotation and journald
• Linux Audit System: Track System Changes – Implement comprehensive system auditing alongside log management
• Bash Scripting Basics: Your First Scripts – Create custom log rotation and monitoring scripts
• Cron vs Anacron vs systemd Timers – Schedule automated log rotation tasks
• Introduction to Ansible for Linux Automation – Automate log rotation across multiple servers
• Git for System Administrators – Version control your logrotate configurations

Upcoming Articles in This Series:

• Backup Strategies: rsync, tar, and Cloud Solutions – Implement backup strategies for archived logs
• System Performance Monitoring with top and htop – Use logs for performance analysis and troubleshooting
• Linux Performance Troubleshooting Methodology – Systematic approaches to identifying performance bottlenecks through log analysis
• Log Analysis with ELK Stack – Build centralized logging infrastructure for enterprise environments
• Log Analysis for Problem Resolution – Master advanced techniques for extracting actionable insights from rotated logs

External Authoritative Resources

Enhance your knowledge with these official documentation sources:

• Logrotate Manual Pages – Complete logrotate reference documentation
• systemd Journal Documentation – Official systemd journal management guide
• Red Hat Log Management Guide – Enterprise log management best practices
• Linux Documentation Project – Comprehensive system administration resources
• Filesystem Hierarchy Standard – Official /var/log structure specification

Conclusion

Effective linux log rotation is fundamental to maintaining healthy, performant Linux systems. By implementing the strategies outlined in this guide, you’ll prevent disk space exhaustion, maintain compliance requirements, and ensure critical diagnostic information remains accessible when needed.

Remember these key principles:

• Start with conservative rotation policies and adjust based on monitoring
• Test configurations in debug mode before production deployment
• Implement monitoring to catch rotation failures early
• Balance retention requirements with storage costs
• Document your rotation policies for future administrators

Master log management, and you’ll gain powerful insights into system behavior while preventing common operational disasters that plague poorly maintained systems.