Command / Code
Bash
Main Command
rsync -avh --progress /home/user/documents/ /backup/documents/Description
How to Use rsync Command in Linux?
Quick Answer: Use rsync -avh source/ destination/ to synchronize files efficiently in Linux. The rsync command only transfers differences between files, making it perfect for backups and file synchronization with minimal bandwidth usage.
Essential rsync Commands (Copy & Paste Ready)
# Basic local sync with progress
rsync -avh --progress /home/user/docs/ /backup/docs/
# Remote backup via SSH
rsync -avz -e ssh /var/www/ user@server:/backup/
# Test sync without making changes
rsync -avh --dry-run /source/ /destination/
# Mirror sync (delete extra files)
rsync -avh --delete /source/ /destination/
# Resume interrupted transfer
rsync -avh --partial --progress /files/ user@remote:/dest/
Frequently Asked Questions
What does rsync -avh mean?
Answer: rsync -avh combines archive mode (-a) for preserving permissions, verbose output (-v) to show progress, and human-readable sizes (-h) for easy reading. It’s the most common rsync command combination.
Is rsync faster than cp command?
Answer: Yes, rsync is faster for updates because it only transfers changed portions of files, while cp copies entire files. For initial copies, cp might be slightly faster locally.
How do I backup files with rsync?
Answer: Use rsync -avh --progress /source/ /backup/ for local backups or rsync -avz -e ssh /local/ user@server:/remote/ for remote backups via SSH.
What’s the difference between /source and /source/?
Answer: /source/ copies the contents of the source directory, while /source copies the directory itself. Always use trailing slashes for consistent behavior.
How do I exclude files in rsync?
Answer: Use --exclude='pattern' to skip specific files. Example: rsync -avh --exclude='*.tmp' --exclude='logs/' /source/ /dest/ excludes temporary files and logs directory.
Can rsync resume interrupted transfers?
Answer: Yes, use rsync -avh --partial --progress to keep partially transferred files and resume them on the next sync attempt.
What is rsync Command in Linux?
The rsync (remote sync) command is a powerful file synchronization tool that efficiently transfers and synchronizes files between directories or systems. Unlike basic copy commands, rsync only transfers the differences between files, making it incredibly fast for large datasets and remote operations.
How to Use rsync Step by Step?
- Basic local sync:
rsync -avh /source/ /destination/ - Remote backup:
rsync -avz /local/ user@server:/remote/ - Test first:
rsync -avh --dry-run /source/ /dest/ - Verify results: Check destination directory for synchronized files
How to Set Up Remote rsync Backup?
- Generate SSH key:
ssh-keygen -t rsa -b 4096 - Copy key to server:
ssh-copy-id user@backup-server - Test connection:
ssh user@backup-server - Run initial backup:
rsync -avz /local/ user@backup-server:/backup/ - Verify files: Check remote directory for transferred files
What are the Most Important rsync Command Examples?
Basic rsync Operations
| Task | Command | Description |
|---|---|---|
| Local backup | rsync -avh /source/ /backup/ | Copy with progress display |
| Remote backup | rsync -avz user@server:/remote/ /local/ | Download from remote server |
| Upload to server | rsync -avz /local/ user@server:/remote/ | Upload to remote server |
| Mirror sync | rsync -avh --delete /source/ /dest/ | Exact copy, delete extra files |
| Test run | rsync -avh --dry-run /source/ /dest/ | Preview without changes |
Basic File Synchronization
# Local directory sync with progress
rsync -avh --progress /home/user/documents/ /backup/documents/
# Remote server backup via SSH
rsync -avz -e ssh /var/www/ user@backup-server:/backups/website/
# Sync with file deletion (mirror)
rsync -avh --delete /source/ /destination/
Advanced rsync Operations
# Test sync without making changes
rsync -avh --dry-run --delete /source/ /destination/
# Exclude specific files and directories
rsync -avh --exclude='*.tmp' --exclude='cache/' /data/ /backup/
# Resume interrupted transfers
rsync -avh --partial --progress /large-files/ user@remote:/destination/
# Bandwidth-limited sync
rsync -avh --bwlimit=1000 /data/ user@remote:/backup/
What are the Most Important rsync Command Options?
| Option | Description | Use Case |
|---|---|---|
-a | Archive mode (preserves permissions, timestamps) | Standard backups |
-v | Verbose output | Monitor transfer progress |
-h | Human-readable file sizes | Easy size reading |
-z | Compress data during transfer | Remote/slow connections |
--delete | Remove files not in source | Mirror synchronization |
--progress | Show transfer progress | Monitor large transfers |
--partial | Keep partial files for resume | Unreliable connections |
--dry-run | Test without making changes | Verify before sync |
--exclude | Skip files matching pattern | Selective synchronization |
When Should You Use rsync Command?
Common rsync Scenarios
| Scenario | Best Command | Why Use rsync |
|---|---|---|
| Daily backups | rsync -avh --delete /data/ /backup/ | Only transfers changes |
| Website deployment | rsync -avz --exclude='.git/' /local/ user@server:/var/www/ | Fast, excludes unnecessary files |
| Large file sync | rsync -avh --partial --progress /files/ /dest/ | Resumable transfers |
| Database backup | rsync -avh /var/lib/mysql/ /backup/mysql/ | Preserves permissions |
| Cross-server migration | rsync -avz -e ssh /old-server/ user@new-server:/ | Secure remote transfer |
How to Create Automated rsync Backup?
- Create backup script:
nano /usr/local/bin/backup.sh - Add rsync command:
rsync -avh --delete /important/ /backup/ - Make executable:
chmod +x /usr/local/bin/backup.sh - Test script:
/usr/local/bin/backup.sh - Add to crontab:
crontab -e - Schedule daily:
0 2 * * * /usr/local/bin/backup.sh
Website Deployment
# Deploy with backup of previous version
rsync -avh --backup --backup-dir=/backup/previous-$(date +%Y%m%d) /staging/ /var/www/html/
Database Backup
# MySQL data directory backup
rsync -avh --inplace /var/lib/mysql/ /backup/mysql-$(date +%Y%m%d)/
Automated Backup Script
#!/bin/bash
# Daily backup with logging
rsync -avh --delete --log-file=/var/log/backup.log /important-data/ /backup/daily/
How to Fix Common rsync Problems?
Step-by-Step rsync Troubleshooting
Problem: Permission denied errors
- Check file permissions:
ls -la /source/ - Verify SSH access:
ssh user@remote-server - Use sudo if needed:
sudo rsync -avh /source/ /dest/ - Fix ownership:
sudo chown -R user:group /destination/
Problem: Slow transfer speeds
- Add compression: Use
-zflag for remote transfers - Limit bandwidth:
rsync -avh --bwlimit=1000 /source/ /dest/ - Check network:
ping remote-serverandtraceroute remote-server - Use different SSH cipher:
rsync -avz -e "ssh -c aes128-ctr" /local/ user@server:/remote/
Problem: Files not syncing as expected
- Test with dry-run:
rsync -avh --dry-run /source/ /dest/ - Check trailing slashes: Ensure consistent
/source/format - Verify exclude patterns: Remove
--excludetemporarily - Check available space:
df -h /destination/
rsync Error Codes Reference
| Exit Code | Meaning | Common Cause |
|---|---|---|
| 0 | Success | Transfer completed successfully |
| 1 | Syntax error | Invalid command syntax |
| 2 | Protocol incompatibility | Version mismatch |
| 3 | File selection errors | Permission issues |
| 5 | Starting transfer error | Connection problems |
| 10 | Socket I/O error | Network connectivity |
| 11 | File I/O error | Disk space or permissions |
| 23 | Partial transfer | Some files couldn’t transfer |
What are the Best rsync Pro Tips?
- Always test first: Use
--dry-runto preview operations before execution - Trailing slash matters:
/source/syncs contents,/sourcesyncs the directory itself - Monitor with stats: Add
--statsfor detailed transfer information - Schedule with cron: Automate regular backups using crontab
- Log operations: Use
--log-filefor audit trails and troubleshooting
How Does rsync Compare to Other Commands?
| Command | Speed | Features | Best For |
|---|---|---|---|
cp | Fast locally | Basic copy | Simple local copies |
scp | Slower | Secure remote copy | One-time remote transfers |
rsync | Fastest for updates | Incremental, resumable | Backups, synchronization |
What are the Best rsync Command Templates?
Quick Copy-Paste rsync Commands
| Purpose | Command Template | Notes |
|---|---|---|
| Basic backup | rsync -avh --progress /source/ /backup/ | Local backup with progress |
| Remote backup | rsync -avz -e ssh /local/ user@server:/remote/ | SSH with compression |
| Mirror sync | rsync -avh --delete /source/ /destination/ | Exact copy, removes extra files |
| Safe test | rsync -avh --dry-run --delete /source/ /dest/ | Preview changes only |
| Resume transfer | rsync -avh --partial --progress /files/ /dest/ | Continue interrupted transfers |
| Exclude files | rsync -avh --exclude='*.tmp' /source/ /dest/ | Skip temporary files |
| Bandwidth limit | rsync -avh --bwlimit=1000 /source/ /dest/ | Limit to 1MB/s |
How to Perform Safe rsync Operation?
- Plan the sync: Identify source and destination directories
- Test first:
rsync -avh --dry-run /source/ /destination/ - Review output: Check which files will be transferred/deleted
- Backup destination:
cp -r /destination/ /destination-backup/ - Run actual sync:
rsync -avh --progress /source/ /destination/ - Verify results:
diff -r /source/ /destination/ - Monitor logs: Check for any error messages or warnings
Mastering the rsync command is essential for efficient file management in Linux. Whether you’re backing up data, deploying websites, or synchronizing directories, rsync provides the speed and reliability needed for professional system administration tasks.
Complete rsync Troubleshooting Guide
How to Diagnose rsync Connection Issues?
SSH Connection Problems
- Test SSH connection:
ssh user@remote-server - Check SSH key:
ssh-add -lto list loaded keys - Verify key on server:
cat ~/.ssh/authorized_keyson remote - Test with password:
rsync -avz -e "ssh -o PasswordAuthentication=yes" /local/ user@server:/remote/ - Check SSH config: Review
/etc/ssh/ssh_configand~/.ssh/config
Network and Firewall Issues
- Test connectivity:
ping remote-server - Check ports:
telnet remote-server 22 - Verify firewall:
sudo ufw statusorsudo iptables -L - Test different port:
rsync -avz -e "ssh -p 2222" /local/ user@server:/remote/
What to Do When rsync Fails?
Disk Space Problems
| Issue | Command to Check | Solution |
|---|---|---|
| Source full | df -h /source/ | Free up space or use different source |
| Destination full | df -h /destination/ | Clean destination or use larger drive |
| Temp space full | df -h /tmp/ | Clean /tmp or set TMPDIR variable |
| Inode exhaustion | df -i /destination/ | Delete unnecessary files or directories |
Permission and Ownership Issues
- Check current permissions:
ls -la /path/to/file - Fix source permissions:
sudo chmod -R 755 /source/ - Fix destination ownership:
sudo chown -R user:group /destination/ - Use numeric IDs:
rsync -avh --numeric-ids /source/ user@server:/dest/ - Skip permission errors:
rsync -rlptgoD --no-perms /source/ /dest/
How to Fix rsync Performance Issues?
Slow Transfer Optimization
- Enable compression: Add
-zflag for remote transfers - Optimize SSH:
rsync -avz -e "ssh -c aes128-ctr -o Compression=no" /local/ user@server:/remote/ - Increase buffer size:
rsync -avz --sockopts=SO_SNDBUF=65536,SO_RCVBUF=65536 /local/ user@server:/remote/ - Use multiple connections: Split large transfers into parallel jobs
- Check network bandwidth:
iperf3 -c remote-server
Memory and CPU Issues
| Problem | Symptom | Solution |
|---|---|---|
| High memory usage | System slow, swap usage | Use --max-size=1G to limit file sizes |
| High CPU usage | 100% CPU utilization | Add nice -n 10 rsync... for lower priority |
| Too many files | “Argument list too long” | Use --files-from with file list |
| Large directories | Very slow startup | Use --no-inc-recursive for faster listing |
Advanced rsync Troubleshooting
Handling Special Characters and File Names
- Unicode issues:
rsync -avz --iconv=utf-8 /source/ /dest/ - Space in names: Use quotes:
rsync -avz "/path with spaces/" /dest/ - Special characters:
rsync -avz --protect-args /source/ /dest/ - Case sensitivity:
rsync -avz --ignore-case /source/ /dest/
Debugging rsync Operations
# Enable verbose debugging
rsync -avvvz --debug=ALL /source/ user@server:/dest/
# Log to file for analysis
rsync -avz --log-file=/var/log/rsync.log /source/ /dest/
# Show statistics
rsync -avz --stats --human-readable /source/ /dest/
# Monitor with progress and itemization
rsync -avz --progress --itemize-changes /source/ /dest/
Common Error Messages and Solutions
| Error Message | Cause | Fix |
|---|---|---|
| “rsync: connection unexpectedly closed” | Network interruption | Retry with --partial flag |
| “rsync error: some files/attrs were not transferred” | Permission issues | Check file permissions and ownership |
| “rsync: failed to connect to server” | SSH/connection problem | Verify SSH keys and network connectivity |
| “rsync: recv_generator: mkdir failed” | Directory creation failed | Check destination permissions |
| “file has vanished” | Source file deleted during sync | Normal for active filesystems |
| “IO error encountered” | Hardware/filesystem issue | Check disk health with fsck |
Emergency rsync Recovery Procedures
Recovering from Failed Transfers
- Resume partial transfer:
rsync -avz --partial /source/ /dest/ - Check for corrupted files:
rsync -avz --checksum /source/ /dest/ - Verify integrity:
find /dest/ -name "*.rsync-partial" -delete - Compare directories:
diff -r /source/ /dest/ - Force complete re-sync:
rsync -avz --ignore-times /source/ /dest/
Data Recovery Best Practices
- Always backup before major syncs:
cp -r /dest/ /dest-backup/ - Use version control:
rsync -avz --backup --backup-dir=/backups/$(date +%Y%m%d) /source/ /dest/ - Test on small dataset first: Verify process works before full sync
- Monitor disk space: Ensure adequate space throughout process
- Keep logs:
--log-filefor audit trail and debugging
Complete rsync Troubleshooting Guide
How to Diagnose rsync Connection Issues?
SSH Connection Problems
- Test SSH connection:
ssh user@remote-server - Check SSH key:
ssh-add -lto list loaded keys - Verify key on server:
cat ~/.ssh/authorized_keyson remote - Test with password:
rsync -avz -e "ssh -o PasswordAuthentication=yes" /local/ user@server:/remote/ - Check SSH config: Review
/etc/ssh/ssh_configand~/.ssh/config
Network and Firewall Issues
- Test connectivity:
ping remote-server - Check ports:
telnet remote-server 22 - Verify firewall:
sudo ufw statusorsudo iptables -L - Test different port:
rsync -avz -e "ssh -p 2222" /local/ user@server:/remote/
What to Do When rsync Fails?
Disk Space Problems
| Issue | Command to Check | Solution |
|---|---|---|
| Source full | df -h /source/ | Free up space or use different source |
| Destination full | df -h /destination/ | Clean destination or use larger drive |
| Temp space full | df -h /tmp/ | Clean /tmp or set TMPDIR variable |
| Inode exhaustion | df -i /destination/ | Delete unnecessary files or directories |
Permission and Ownership Issues
- Check current permissions:
ls -la /path/to/file - Fix source permissions:
sudo chmod -R 755 /source/ - Fix destination ownership:
sudo chown -R user:group /destination/ - Use numeric IDs:
rsync -avh --numeric-ids /source/ user@server:/dest/ - Skip permission errors:
rsync -rlptgoD --no-perms /source/ /dest/
How to Fix rsync Performance Issues?
Slow Transfer Optimization
- Enable compression: Add
-zflag for remote transfers - Optimize SSH:
rsync -avz -e "ssh -c aes128-ctr -o Compression=no" /local/ user@server:/remote/ - Increase buffer size:
rsync -avz --sockopts=SO_SNDBUF=65536,SO_RCVBUF=65536 /local/ user@server:/remote/ - Use multiple connections: Split large transfers into parallel jobs
- Check network bandwidth:
iperf3 -c remote-server
Memory and CPU Issues
| Problem | Symptom | Solution |
|---|---|---|
| High memory usage | System slow, swap usage | Use --max-size=1G to limit file sizes |
| High CPU usage | 100% CPU utilization | Add nice -n 10 rsync... for lower priority |
| Too many files | “Argument list too long” | Use --files-from with file list |
| Large directories | Very slow startup | Use --no-inc-recursive for faster listing |
Advanced rsync Troubleshooting
Handling Special Characters and File Names
- Unicode issues:
rsync -avz --iconv=utf-8 /source/ /dest/ - Space in names: Use quotes:
rsync -avz "/path with spaces/" /dest/ - Special characters:
rsync -avz --protect-args /source/ /dest/ - Case sensitivity:
rsync -avz --ignore-case /source/ /dest/
Debugging rsync Operations
# Enable verbose debugging
rsync -avvvz --debug=ALL /source/ user@server:/dest/
# Log to file for analysis
rsync -avz --log-file=/var/log/rsync.log /source/ /dest/
# Show statistics
rsync -avz --stats --human-readable /source/ /dest/
# Monitor with progress and itemization
rsync -avz --progress --itemize-changes /source/ /dest/
Common Error Messages and Solutions
| Error Message | Cause | Fix |
|---|---|---|
| “rsync: connection unexpectedly closed” | Network interruption | Retry with --partial flag |
| “rsync error: some files/attrs were not transferred” | Permission issues | Check file permissions and ownership |
| “rsync: failed to connect to server” | SSH/connection problem | Verify SSH keys and network connectivity |
| “rsync: recv_generator: mkdir failed” | Directory creation failed | Check destination permissions |
| “file has vanished” | Source file deleted during sync | Normal for active filesystems |
| “IO error encountered” | Hardware/filesystem issue | Check disk health with fsck |
Emergency rsync Recovery Procedures
Recovering from Failed Transfers
- Resume partial transfer:
rsync -avz --partial /source/ /dest/ - Check for corrupted files:
rsync -avz --checksum /source/ /dest/ - Verify integrity:
find /dest/ -name "*.rsync-partial" -delete - Compare directories:
diff -r /source/ /dest/ - Force complete re-sync:
rsync -avz --ignore-times /source/ /dest/
Data Recovery Best Practices
- Always backup before major syncs:
cp -r /dest/ /dest-backup/ - Use version control:
rsync -avz --backup --backup-dir=/backups/$(date +%Y%m%d) /source/ /dest/ - Test on small dataset first: Verify process works before full sync
- Monitor disk space: Ensure adequate space throughout process
- Keep logs:
--log-filefor audit trail and debugging
Detailed Explanation
The Linux command rsync -avh –progress /home/user/documents/ /backup/documents/ is used to synchronize files and directories between two locations, in this case from /home/user/documents/ to /backup/documents/. Here’s a breakdown of each component:
- rsync: A powerful tool for copying and synchronizing files and directories, either locally or between remote systems. It efficiently transfers only the differences between source and destination, making it faster than a full copy.
- -a (archive mode): Enables a set of options to preserve file attributes during the transfer, including:
- File permissions
- Timestamps
- Symbolic links
- Ownership
- Group information
- Recursively copies directories This ensures the destination files are as close as possible to the source in terms of metadata.
- -v (verbose): Provides detailed output about the files being transferred, showing which files are being copied or skipped.
- -h (human-readable): Formats file sizes in the output (e.g., KB, MB, GB) to be easier to read.
- –progress: Shows the progress of each file transfer, including the percentage completed, transfer speed, and estimated time remaining.
- /home/user/documents/: The source directory. The trailing slash (/) means rsync copies the contents of the documents directory, not the directory itself.
- /backup/documents/: The destination directory where the files will be copied. If it doesn’t exist, rsync will create it.
What the Command DoesThis command copies all files and directories from /home/user/documents/ to /backup/documents/, preserving file attributes (permissions, timestamps, etc.), showing detailed output with progress information, and displaying file sizes in a human-readable format. If files already exist in the destination, rsync will only transfer the differences (e.g., updated or new files), making it efficient for backups or synchronization.Example OutputWhen you run the command, you might see something like:
building file list ... done
file1.txt
1.00K 100% 1.19MB/s 00:00:00 (xfr#1, to-chk=99/100)
file2.txt
10.00M 100% 5.00MB/s 00:00:02 (xfr#2, to-chk=98/100)
sent 10.01M bytes received 92 bytes 2.00M bytes/sec
total size is 10.00M speedup is 1.00Notes
- If you omit the trailing slash on the source (/home/user/documents), rsync will copy the documents directory itself into /backup/documents/documents/, which may not be what you want.
- Ensure the destination path (/backup/documents/) has sufficient permissions and space.
- Rsync is highly efficient for incremental backups, as it only transfers changes.
If you need more details or have a specific use case, let me know!