💻
🔧
🐧
Beginner
Bash
July 16, 2025

Linux journalctl Command: Find Timeout Errors Tutorial

Description

How to Use Linux journalctl Command for Log Analysis?

Quick Answer: Use journalctl -b to view current boot logs, journalctl -f to follow live logs, and journalctl -u servicename to check specific services. Filter with grep for errors: journalctl -b | grep "error". Essential for systemd log management and troubleshooting.

Essential journalctl Command Examples

# View current boot logs
journalctl -b

# Follow live logs (like tail -f)
journalctl -f

# Check specific service logs
journalctl -u nginx.service

# Show only error messages
journalctl -p err

# Find timeout errors
journalctl -b | grep "Timed out waiting"

# View logs from last hour
journalctl --since "1 hour ago"

# Show last 50 entries
journalctl -n 50

# Check kernel messages
journalctl -k

# View logs in reverse order (newest first)
journalctl -r

# Filter by time range
journalctl --since "2025-01-15 09:00:00" --until "2025-01-15 17:00:00"

# Follow specific service logs live
journalctl -u ssh.service -f

# Find failed services
journalctl -b | grep "Failed to start"

Frequently Asked Questions

Q: What does journalctl command do in Linux? A: journalctl is the systemd journal query tool that displays and filters system logs. It replaces traditional log file viewing and provides centralized access to all system, kernel, and service logs.

Q: How do I view current boot logs with journalctl? A: Use journalctl -b to show logs from the current boot session only, or journalctl -b -1 for the previous boot. This helps isolate current session issues from historical problems.

Q: How do I follow live logs like tail -f? A: Use journalctl -f to follow logs in real-time, similar to tail -f. Add -u servicename to follow a specific service: journalctl -u nginx -f.

Q: How do I check logs for a specific service? A: Use journalctl -u servicename to view logs for a specific systemd service. Example: journalctl -u ssh.service shows SSH service logs.

Q: How do I filter journalctl by time? A: Use --since and --until options with time expressions: journalctl --since "1 hour ago" or journalctl --since "2025-01-15 09:00:00".

Essential Steps to Master journalctl Log Analysis

  1. Start with boot logs: Use journalctl -b to view current session logs and identify system startup issues
  2. Check service status: Use journalctl -u servicename to investigate specific service problems and failures
  3. Filter by priority: Use journalctl -p err to focus on error-level messages and critical issues
  4. Use time filters: Apply --since and --until to narrow down logs to specific time periods
  5. Follow live logs: Use journalctl -f for real-time monitoring during troubleshooting sessions
  6. Combine with grep: Pipe output to grep for specific patterns: journalctl -b | grep "error"

Most Important journalctl Commands

CommandPurposeUse Case
journalctl -bCurrent boot logsSystem startup issues
journalctl -fFollow live logsReal-time monitoring
journalctl -u serviceService-specific logsService troubleshooting
journalctl -p errError messages onlyCritical issue detection
journalctl --since "1h ago"Time-filtered logsRecent problem analysis

What Are the Most Important journalctl Command Options?

Essential Viewing Options

# Current boot session logs
journalctl -b

# Previous boot session logs
journalctl -b -1

# Follow logs in real-time
journalctl -f

# Show last N entries
journalctl -n 50

# Reverse order (newest first)
journalctl -r

Service and Unit Filtering

# Specific service logs
journalctl -u nginx.service

# Multiple services
journalctl -u nginx.service -u mysql.service

# All systemd units
journalctl -u "*.service"

# Kernel messages only
journalctl -k

How Do You Filter journalctl Logs by Time?

Time-Based Filtering Options

# Logs from last hour
journalctl --since "1 hour ago"

# Logs from last 30 minutes
journalctl --since "30 minutes ago"

# Logs from today
journalctl --since today

# Logs from yesterday
journalctl --since yesterday

# Logs from specific date
journalctl --since "2025-01-15"

Advanced Time Filtering

# Specific date and time
journalctl --since "2025-01-15 09:00:00"

# Time range filtering
journalctl --since "2 hours ago" --until "1 hour ago"

# Date range filtering
journalctl --since "2025-01-15" --until "2025-01-16"

# Combine with other filters
journalctl -u nginx.service --since "1 hour ago"

Time Format Examples

# Relative times
journalctl --since "2 days ago"
journalctl --since "1 week ago"
journalctl --since "10 minutes ago"

# Absolute times
journalctl --since "2025-01-15 14:30:00"
journalctl --since "Jan 15 14:30:00"
journalctl --since "15:30"

How Do You Filter journalctl by Priority and Message Type?

Priority Level Filtering

# Emergency messages only
journalctl -p emerg

# Alert level and above
journalctl -p alert

# Critical errors and above
journalctl -p crit

# Error messages and above
journalctl -p err

# Warning messages and above
journalctl -p warning

# Informational and above
journalctl -p info

# Debug messages (all)
journalctl -p debug

Message Type Filtering

# Kernel messages only
journalctl -k

# User session messages
journalctl --user

# System messages only
journalctl --system

# Messages from specific process
journalctl _PID=1234

# Messages from specific executable
journalctl _COMM=sshd

Advanced Pattern Filtering

# Find timeout errors
journalctl -b | grep "Timed out waiting"

# Find failed services
journalctl -b | grep "Failed to start"

# Find memory errors
journalctl -k | grep -i "memory"

# Find disk errors
journalctl -k | grep -i "disk"

# Find network errors
journalctl | grep -E "(network|eth0|connection)"

What Are Advanced journalctl Command Techniques?

Output Format Options

# JSON output for parsing
journalctl -o json

# JSON output (pretty printed)
journalctl -o json-pretty

# Short format (syslog style)
journalctl -o short

# Verbose format (all fields)
journalctl -o verbose

# Export format for backup
journalctl -o export

# CAT format (message only)
journalctl -o cat

Field-Based Filtering

# Filter by specific systemd unit
journalctl _SYSTEMD_UNIT=nginx.service

# Filter by process ID
journalctl _PID=1234

# Filter by user ID
journalctl _UID=1000

# Filter by executable name
journalctl _COMM=sshd

# Filter by hostname
journalctl _HOSTNAME=server01

Disk Usage and Maintenance

# Check journal disk usage
journalctl --disk-usage

# Verify journal integrity
journalctl --verify

# Vacuum old logs (keep 1 week)
sudo journalctl --vacuum-time=1week

# Vacuum by size (keep 1GB)
sudo journalctl --vacuum-size=1G

# Vacuum by number of files
sudo journalctl --vacuum-files=10

When Should You Use Different journalctl Commands?

System Troubleshooting Scenarios

# Boot problems - check current boot
journalctl -b -p err

# Service failures - check specific service
journalctl -u failed-service.service -n 100

# Performance issues - check recent system logs
journalctl --since "1 hour ago" -p warning

# Hardware problems - check kernel messages
journalctl -k --since "1 hour ago"

Development and Debugging

# Application debugging - follow app logs
journalctl -u myapp.service -f

# Web server issues - check access patterns
journalctl -u nginx.service --since "30 minutes ago"

# Database problems - monitor database service
journalctl -u mysql.service -p err

# Container issues - check container runtime
journalctl -u docker.service --since today

Security Analysis

# Authentication failures
journalctl | grep "authentication failure"

# SSH login attempts
journalctl -u ssh.service | grep "Failed password"

# Sudo usage tracking
journalctl | grep "sudo.*COMMAND"

# System security events
journalctl -p warning | grep -E "(security|auth|login)"

What Are All journalctl Command Options?

OptionDescriptionExample
-bBoot session logsjournalctl -b
-fFollow live logsjournalctl -f
-uSpecific unit/servicejournalctl -u nginx
-pPriority leveljournalctl -p err
-nNumber of entriesjournalctl -n 50
-rReverse orderjournalctl -r
-kKernel messagesjournalctl -k
-xAdd explanatory textjournalctl -x
--sinceTime filter startjournalctl --since "1h ago"
--untilTime filter endjournalctl --until "now"

What Are Essential journalctl Log Analysis Practices?

Daily Monitoring Routine

# Check for critical errors
journalctl -p crit --since today

# Review service failures
journalctl -b | grep "Failed to start"

# Monitor system performance
journalctl -k --since "1 hour ago" | grep -E "(error|warning)"

# Check authentication logs
journalctl --since today | grep -E "(login|auth)"

Automated Log Analysis

# Create daily error report
journalctl -p err --since today > /tmp/daily_errors.log

# Monitor specific service health
journalctl -u critical-service.service --since "5 minutes ago" -p warning

# Check for disk issues
journalctl -k | grep -E "(disk|filesystem|mount)" | tail -20

# Network connectivity monitoring
journalctl --since "10 minutes ago" | grep -E "(network|connection|timeout)"

Performance Optimization

# Limit journal size
sudo journalctl --vacuum-size=100M

# Clean old entries
sudo journalctl --vacuum-time=2weeks

# Persistent storage configuration
sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

How Can You Create Useful journalctl Aliases and Scripts?

Essential Aliases

# Add to ~/.bashrc
alias jb='journalctl -b'                    # Current boot
alias jf='journalctl -f'                    # Follow logs
alias je='journalctl -p err'                # Errors only
alias jt='journalctl --since "1 hour ago"' # Last hour
alias jk='journalctl -k'                    # Kernel only

Advanced Functions

# Service log checker
jservice() {
    if [ -z "$1" ]; then
        echo "Usage: jservice <service-name>"
        return 1
    fi
    journalctl -u "$1.service" -n 50 --no-pager
}

# Error finder with context
jerrors() {
    local since=${1:-"1 hour ago"}
    echo "=== Errors since $since ==="
    journalctl --since "$since" -p err --no-pager
}

# Boot analysis
jboot() {
    local boot=${1:-0}
    echo "=== Boot Analysis (boot $boot) ==="
    journalctl -b "$boot" -p warning --no-pager | head -20
}

# Service monitor
jmonitor() {
    if [ -z "$1" ]; then
        echo "Usage: jmonitor <service-name>"
        return 1
    fi
    echo "Monitoring $1.service (Ctrl+C to stop)..."
    journalctl -u "$1.service" -f --no-pager
}

Diagnostic Scripts

#!/bin/bash
# system_health_check.sh - Comprehensive log analysis

echo "=== System Health Check $(date) ==="

echo -e "\n1. Critical Errors:"
journalctl -p crit --since today --no-pager | head -10

echo -e "\n2. Failed Services:"
journalctl -b | grep "Failed to start" --no-pager | head -10

echo -e "\n3. Recent Timeouts:"
journalctl -b | grep "Timed out waiting" --no-pager | head -10

echo -e "\n4. Kernel Warnings:"
journalctl -k --since "1 hour ago" -p warning --no-pager | head -10

echo -e "\n5. Authentication Failures:"
journalctl --since today | grep "authentication failure" --no-pager | head -10

echo -e "\n6. Disk Usage:"
journalctl --disk-usage

What Commands Are Related to journalctl?

  • systemctl – Control systemd services and view status
  • dmesg – Display kernel ring buffer messages
  • tail – View end of files (traditional log viewing)
  • grep – Search text patterns in logs
  • awk – Advanced text processing and filtering
  • less – Paginated viewing of log files
  • logrotate – Manage log file rotation
  • rsyslog – Traditional system logging daemon
  • systemd-analyze – Analyze system boot performance

Common journalctl Command Problems and Solutions

“No journal files were found” Error

Problem: journalctl shows no logs or journal files missing

Diagnosis:

# Check if systemd-journald is running
systemctl status systemd-journald

# Check journal directory
ls -la /var/log/journal/

# Check for runtime journal
ls -la /run/log/journal/

# Check journal configuration
cat /etc/systemd/journald.conf

Solutions:

# Create persistent journal directory
sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

# Enable persistent storage
sudo sed -i 's/#Storage=auto/Storage=persistent/' /etc/systemd/journald.conf
sudo systemctl restart systemd-journald

# Check disk space
df -h /var/log

# Fix permissions
sudo chown root:systemd-journal /var/log/journal
sudo chmod 2755 /var/log/journal

“Failed to determine timestamp” Error

Problem: Corrupted journal files or timestamp issues

Diagnosis:

# Verify journal integrity
journalctl --verify

# Check for corrupted files
sudo journalctl --verify --quiet

# Check system time
timedatectl status

# Check for clock synchronization
systemctl status systemd-timesyncd

Solutions:

# Rebuild journal database
sudo systemctl stop systemd-journald
sudo rm /var/log/journal/*/*
sudo systemctl start systemd-journald

# Fix system time
sudo timedatectl set-ntp true
sudo timedatectl set-timezone America/New_York

# Vacuum corrupted entries
sudo journalctl --vacuum-time=1d

# Force journal rotation
sudo systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald

Permission Denied Accessing Logs

Problem: Cannot view system logs as regular user

Diagnosis:

# Check current user groups
groups

# Check journal file permissions
ls -la /var/log/journal/

# Check systemd-journal group membership
getent group systemd-journal

# Test with sudo
sudo journalctl -b

Solutions:

# Add user to systemd-journal group
sudo usermod -a -G systemd-journal $USER

# Re-login or use newgrp
newgrp systemd-journal

# Use sudo for system logs
sudo journalctl -b

# Check adm group membership (alternative)
sudo usermod -a -G adm $USER

journalctl Command Hanging

Problem: journalctl command appears to hang or freeze

Diagnosis:

# Check if pager is active
ps aux | grep less

# Check journal size
journalctl --disk-usage

# Check system load
top | head -5

# Check for large journal files
find /var/log/journal -type f -size +100M

Solutions:

# Disable pager
journalctl --no-pager

# Use head to limit output
journalctl -n 100

# Vacuum large journals
sudo journalctl --vacuum-size=100M

# Use time limits
journalctl --since "1 hour ago"

# Kill hanging process
pkill -f journalctl

Logs Not Appearing in Real Time

Problem: Live log following (-f) not showing new entries

Diagnosis:

# Check if service is active
systemctl is-active systemd-journald

# Check if logs are being written
ls -la /var/log/journal/*/system.journal

# Test with different service
journalctl -u systemd-journald -f

# Check inotify limits
cat /proc/sys/fs/inotify/max_user_watches

Solutions:

# Restart journald service
sudo systemctl restart systemd-journald

# Increase inotify limits
echo 'fs.inotify.max_user_watches=524288' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Use polling mode
journalctl -f --interval=1

# Check specific service
journalctl -u your-service.service -f

High Memory Usage by journald

Problem: systemd-journald consuming excessive memory

Diagnosis:

# Check journald memory usage
ps aux | grep systemd-journald

# Check journal size
journalctl --disk-usage

# Check rate limiting
journalctl | grep "rate"

# Monitor memory in real-time
top -p $(pgrep systemd-journald)

Solutions:

# Configure journal size limits
sudo tee /etc/systemd/journald.conf << EOF
[Journal]
SystemMaxUse=100M
RuntimeMaxUse=50M
MaxFileSec=1week
MaxRetentionSec=1month
EOF

# Restart journald
sudo systemctl restart systemd-journald

# Vacuum old logs
sudo journalctl --vacuum-size=50M
sudo journalctl --vacuum-time=1week

# Enable rate limiting
sudo tee -a /etc/systemd/journald.conf << EOF
RateLimitInterval=30s
RateLimitBurst=10000
EOF

Missing Service Logs

Problem: Specific service logs not appearing in journal

Diagnosis:

# Check if service is using journal
systemctl status your-service.service

# Check service configuration
systemctl cat your-service.service

# Look for syslog configuration
grep -r "syslog" /etc/your-service/

# Check for custom logging
ps aux | grep your-service

Solutions:

# Ensure service uses systemd logging
sudo systemctl edit your-service.service
# Add:
# [Service]
# StandardOutput=journal
# StandardError=journal

# Check for service-specific logs
tail -f /var/log/your-service.log

# Force service to use journal
sudo systemctl daemon-reload
sudo systemctl restart your-service.service

# Check journal configuration
journalctl -u your-service.service --since "10 minutes ago"

Journal File Corruption

Problem: Corrupted journal files preventing log access

Diagnosis:

# Verify journal integrity
journalctl --verify

# Check for specific errors
sudo journalctl --verify 2>&1 | grep -i error

# Identify corrupted files
find /var/log/journal -name "*.journal" -exec journalctl --file {} --verify \;

# Check disk health
sudo fsck /var

Solutions:

# Backup existing journals
sudo cp -r /var/log/journal /var/log/journal.backup

# Remove corrupted files (WARNING: data loss)
sudo rm /var/log/journal/*/system@*.journal~

# Force journal rotation
sudo systemctl kill --signal=SIGUSR2 systemd-journald

# Rebuild journal completely (last resort)
sudo systemctl stop systemd-journald
sudo rm -rf /var/log/journal/*
sudo systemctl start systemd-journald

# Restore from backup if needed
sudo cp -r /var/log/journal.backup/* /var/log/journal/

Mastering the journalctl command enables efficient system log analysis, troubleshooting, and monitoring in modern Linux environments using systemd’s centralized logging system.

Detailed Explanation

This Linux command searches system logs for timeout-related error messages. journalctl -b journalctl is the command to view systemd journal logs (the modern Linux logging system) -b flag shows logs from the current boot session only (since the last system startup) | This is a pipe operator that takes the output from the left command and feeds it as input to the right command grep "Timed out waiting" grep searches through text for specific patterns "Timed out waiting" is the search pattern - it will find any log entries containing this exact phrase What it does: The command displays all log entries from the current boot that contain the text "Timed out waiting". This is useful for troubleshooting because timeout errors often indicate: Services that failed to start within their allocated time Network operations that couldn't complete Hardware devices that didn't respond File system operations that hung Example output might look like: Jan 15 10:23:45 hostname systemd[1]: Timed out waiting for device /dev/disk/by-uuid/... Jan 15 10:24:12 hostname systemd[1]: Timed out waiting for network interface This is a common diagnostic command when investigating system startup issues or performance problems.

Related Commands