Git Version Control for System Administrators Linux Mastery Series
Prerequisites
Why Do System Administrators Need Git Version Control?
Git version control enables system administrators to track every configuration change, rollback problematic updates instantly, collaborate with teams safely, and maintain comprehensive audit trails of infrastructure modifications. Unlike traditional backup methods, Git provides granular version history, branching for testing changes, and distributed repositories ensuring your configurations are never lost.
Quick Win Example – Track /etc Configurations:
# Initialize Git repository for /etc
cd /etc
sudo git init
# Add all configuration files
sudo git add .
# Create initial snapshot
sudo git commit -m "Initial /etc configuration snapshot - $(date +%Y-%m-%d)"
# Make a change to nginx config
sudo vi nginx/nginx.conf
# See what changed
sudo git diff
# Stage and commit the change
sudo git add nginx/nginx.conf
sudo git commit -m "Increased worker_processes to 4"
# View change history
sudo git log --oneline
# Rollback if needed
sudo git revert HEAD
This immediately demonstrates Git’s power: every configuration change is tracked, documented, and reversible, transforming chaotic configuration management into controlled, auditable infrastructure evolution.
Table of Contents
- How Does Git Version Control Work for System Administration?
- What Are the Benefits of Git for Infrastructure Management?
- How to Install and Configure Git on Linux?
- How to Set Up Your First Configuration Repository?
- What Files Should System Administrators Track with Git?
- How to Use Git Branching for Safe Testing?
- How to Collaborate with Teams Using Git?
- What Are Git Hooks and How to Automate Workflows?
- How to Implement Infrastructure as Code with Git?
- Best Practices for Git Version Control in Production
- Frequently Asked Questions
- Troubleshooting Common Git Issues
How Does Git Version Control Work for System Administration?
Git transforms configuration management from manual tracking to automated version history. Consequently, understanding Git’s architecture enables system administrators to implement robust change control across their infrastructure.
Git’s Three-Stage Architecture
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Working Directory β
β Your actual configuration files β
β /etc/nginx/nginx.conf β
β /etc/mysql/my.cnf β
β scripts/backup.sh β
ββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ
β git add
βΌ
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Staging Area (Index) β
β Changes prepared for commit β
β + nginx.conf (modified) β
β + backup.sh (new file) β
ββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ
β git commit
βΌ
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Local Repository (.git/) β
β Complete history of all commits β
β Commit abc123: "Update nginx config" β
β Commit def456: "Add backup script" β
ββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ
β git push
βΌ
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Remote Repository β
β GitHub/GitLab/Bitbucket or self-hosted β
β Shared with team, backed up externally β
ββββββββββββββββββββββββββββββββββββββββββββββββββββ
Understanding Git Objects
# View Git object types
cd /etc/.git
# Commits - snapshots of your configurations
git log --oneline
# abc123 Update nginx worker processes
# def456 Add MySQL slow query log
# ghi789 Configure firewall rules
# Blobs - file contents
git show abc123:nginx/nginx.conf
# Trees - directory structure
git ls-tree HEAD
# Tags - marked releases
git tag -a v1.0 -m "Production configuration v1.0"
Practical Example – Track Server Configuration Changes:
#!/bin/bash
# /usr/local/bin/track-config-change.sh
# Automatically commit configuration changes with context
CONFIG_REPO="/etc"
CHANGE_DESC="$1"
cd "$CONFIG_REPO"
# Check if there are changes
if ! git diff-index --quiet HEAD --; then
# Show what changed
echo "Configuration changes detected:"
git status --short
# Commit with timestamp and description
git add -A
git commit -m "${CHANGE_DESC} - $(hostname) - $(date '+%Y-%m-%d %H:%M:%S')"
# Push to remote backup
git push origin main
echo "Changes committed and backed up"
else
echo "No configuration changes detected"
fi
Moreover, the Bash Scripting Basics guide explains how to create automation scripts that integrate with Git workflows.
What Are the Benefits of Git for Infrastructure Management?
Implementing git version control for infrastructure management provides tangible advantages over traditional configuration backup methods. Therefore, understanding these benefits helps justify adoption across your organization.
Comparison: Traditional vs Git-Based Management
Aspect | Traditional Backups | Git Version Control |
---|---|---|
Granularity | Full file copies | Line-by-line changes |
History | Date-stamped archives | Complete change history |
Rollback | Restore entire file | Selective line reversion |
Collaboration | Manual merging | Automated merge tools |
Audit Trail | Filename timestamps | Commit messages, authors |
Testing | Copy entire directory | Branches for safe testing |
Storage | Grows linearly | Compressed delta storage |
Recovery | Single point failure | Distributed copies |
Real-World Benefit Examples
1. Instant Rollback After Problematic Change
# Scenario: Nginx update breaks production
sudo systemctl status nginx
# nginx.service - failed
# View recent changes
cd /etc/nginx
sudo git log --oneline -5
# abc123 Increase worker_connections to 10000
# def456 Add gzip compression
# ghi789 Update SSL protocols
# Identify the breaking commit
sudo git show abc123
# Rollback the specific change
sudo git revert abc123
# Or rollback multiple commits
sudo git reset --hard ghi789
# Restart service
sudo systemctl restart nginx
sudo systemctl status nginx
# nginx.service - active (running)
# Total recovery time: < 2 minutes
2. Team Collaboration Without Conflicts
# Admin 1 works on nginx configuration
cd /etc
sudo git checkout -b feature/nginx-optimization
sudo vi nginx/nginx.conf
sudo git commit -am "Optimize nginx for high traffic"
sudo git push origin feature/nginx-optimization
# Admin 2 works on MySQL simultaneously
sudo git checkout -b feature/mysql-tuning
sudo vi mysql/my.cnf
sudo git commit -am "Increase InnoDB buffer pool"
sudo git push origin feature/mysql-tuning
# Both changes merged without conflict
sudo git checkout main
sudo git merge feature/nginx-optimization
sudo git merge feature/mysql-tuning
# No configuration overwrites or lost changes
3. Comprehensive Audit Trail
# Who changed the firewall rules?
cd /etc
sudo git log --all --full-history -- ufw/user.rules
# commit abc123
# Author: admin1 <admin1@company.com>
# Date: 2025-10-01 14:30:00
#
# Open port 8080 for application server
# Ticket: OPS-1234
# What was the SSH configuration 3 months ago?
sudo git show HEAD@{3.months.ago}:ssh/sshd_config
# All changes by specific user
sudo git log --author="admin2"
# Changes during specific time period
sudo git log --since="2025-09-01" --until="2025-10-01" \
--pretty=format:"%h %an %ad %s" --date=short
4. Safe Testing Before Production
# Test risky Apache configuration change
cd /etc/apache2
sudo git checkout -b test/new-vhost-config
# Make experimental changes
sudo vi sites-available/new-app.conf
sudo git add sites-available/new-app.conf
sudo git commit -m "Test: New application vhost"
# Test the configuration
sudo apachectl configtest
# If successful, merge to main
sudo git checkout main
sudo git merge test/new-vhost-config
# If failed, simply delete branch
sudo git branch -D test/new-vhost-config
# No production impact
Furthermore, the Linux Security Essentials guide demonstrates how Git’s audit capabilities complement security monitoring.
How to Install and Configure Git on Linux?
Setting up git version control correctly from the start ensures smooth infrastructure management. Consequently, proper initial configuration prevents common issues and establishes team conventions.
Installation Across Major Distributions
# Ubuntu/Debian
sudo apt update
sudo apt install -y git
# RHEL/CentOS/Rocky Linux
sudo dnf install -y git
# or for older versions
sudo yum install -y git
# Fedora
sudo dnf install -y git
# Arch Linux
sudo pacman -S git
# Verify installation
git --version
# git version 2.40.0 or newer
Essential Global Configuration
# Set your identity (appears in commits)
git config --global user.name "John Admin"
git config --global user.email "jadmin@company.com"
# Set default branch name
git config --global init.defaultBranch main
# Set default editor
git config --global core.editor vim
# Enable color output
git config --global color.ui auto
# Configure line endings (Unix)
git config --global core.autocrlf input
# Set up credential caching (1 hour)
git config --global credential.helper 'cache --timeout=3600'
# Configure merge strategy
git config --global pull.rebase false
# Enable fsck on receive
git config --global transfer.fsckObjects true
# View all configuration
git config --list --show-origin
System-Wide Configuration for Servers
# Create system-wide Git configuration
sudo tee /etc/gitconfig << 'EOF'
# Require explicit user configuration useConfigOnly = true
[core]
# Protect against CRLF injection autocrlf = input # Prevent file permission issues fileMode = false # Use system pager pager = less -R
[commit]
# Require commit messages template = /etc/git-commit-template.txt
[color]
ui = auto status = auto branch = auto
[alias]
# Useful shortcuts st = status -s co = checkout br = branch ci = commit unstage = reset HEAD — last = log -1 HEAD visual = log –graph –oneline –all EOF # Create commit message template sudo tee /etc/git-commit-template.txt << ‘EOF’ # [TYPE] Brief description (max 50 chars) # # Detailed explanation (wrap at 72 chars): # – Why is this change necessary? # – How does it address the issue? # – What are the side effects? # # References: Ticket #, Related commits # # Types: CONFIG, SCRIPT, SECURITY, UPDATE, FIX EOF
SSH Key Configuration for Git
# Generate SSH key for Git operations
ssh-keygen -t ed25519 -C "git@$(hostname)" -f ~/.ssh/id_git_ed25519 -N ""
# Add to SSH agent
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_git_ed25519
# Display public key for adding to GitHub/GitLab
cat ~/.ssh/id_git_ed25519.pub
# Configure SSH for Git hosts
mkdir -p ~/.ssh
tee -a ~/.ssh/config << 'EOF'
# GitHub
Host github.com
HostName github.com
User git
IdentityFile ~/.ssh/id_git_ed25519
IdentitiesOnly yes
# GitLab
Host gitlab.com
HostName gitlab.com
User git
IdentityFile ~/.ssh/id_git_ed25519
IdentitiesOnly yes
# Company GitLab
Host gitlab.company.local
HostName gitlab.company.local
User git
IdentityFile ~/.ssh/id_git_ed25519
IdentitiesOnly yes
Port 2222
EOF
# Test connectivity
ssh -T git@github.com
# Hi username! You've successfully authenticated
Git Prompt for Bash (Optional but Recommended)
# Add Git branch to bash prompt
cat >> ~/.bashrc << 'EOF'
# Git branch in prompt
parse_git_branch() {
git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'
}
# Update PS1
if [ "$color_prompt" = yes ]; then
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[01;31m\]$(parse_git_branch)\[\033[00m\]\$ '
else
PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w$(parse_git_branch)\$ '
fi
EOF
# Reload bash configuration
source ~/.bashrc
# Prompt now shows: user@host:/etc(main)$
Additionally, the SSH Server Setup guide covers secure key management practices applicable to Git workflows.
How to Set Up Your First Configuration Repository?
Creating your first git version control repository for configuration management requires careful planning. Therefore, following a systematic approach ensures clean history and maintainability.
1: Track /etc Directory
# Initialize Git in /etc (most common approach)
cd /etc
sudo git init
# Create .gitignore for sensitive files
sudo tee .gitignore << 'EOF'
# Ignore sensitive credential files
shadow
shadow-
gshadow
gshadow-
passwd-
group-
# Ignore SSL private keys
ssl/private/*
*.key
*.pem
# Ignore dynamic/runtime files
*.pid
*.sock
*.lock
mtab
resolv.conf
# Ignore package manager files
apt/trusted.gpg.d/*
yum.repos.d/*.repo
# Ignore large binary files
*.db
*.cache
EOF
# Set proper permissions
sudo chown -R root:root .git
sudo chmod 700 .git
# Initial commit
sudo git add .
sudo git commit -m "Initial /etc configuration snapshot - $(hostname) - $(date -I)"
# Tag as baseline
sudo git tag -a baseline-$(date +%Y%m%d) -m "Baseline configuration"
# View what was added
sudo git log --stat
2: Selective Configuration Repository
# Create dedicated configuration repository
sudo mkdir -p /opt/config-repo
cd /opt/config-repo
sudo git init
# Create organized structure
sudo mkdir -p {nginx,apache,mysql,ssh,firewall,scripts,cron}
# Copy configurations (preserving originals)
sudo cp -a /etc/nginx/nginx.conf nginx/
sudo cp -a /etc/nginx/sites-available nginx/sites-available
sudo cp -a /etc/ssh/sshd_config ssh/
sudo cp -a /etc/mysql/my.cnf mysql/
sudo cp -a /etc/ufw/*.rules firewall/
# Copy custom scripts
sudo cp -a /usr/local/bin/*.sh scripts/
# Copy cron jobs
sudo cp -a /etc/cron.d/* cron/
# Create README
sudo tee README.md << 'EOF'
# Server Configuration Repository
## Purpose
Track and version control critical server configurations
## Structure
- `nginx/` - Nginx web server configuration
- `apache/` - Apache configuration
- `mysql/` - MySQL/MariaDB configuration
- `ssh/` - SSH daemon configuration
- `firewall/` - UFW/iptables rules
- `scripts/` - Custom administration scripts
- `cron/` - Scheduled tasks
## Usage
1. Make changes to files in this repository
2. Test changes thoroughly
3. Deploy using: `sudo ./deploy.sh`
4. Commit with descriptive message
## Server Info
- Hostname: $(hostname)
- OS: $(cat /etc/os-release | grep PRETTY_NAME | cut -d'"' -f2)
- Last Updated: $(date)
EOF
# Initial commit
sudo git add .
sudo git commit -m "Initial configuration repository"
# Create deployment script
sudo tee deploy.sh << 'EOF'
#!/bin/bash
# Deploy configurations from repository to system
set -euo pipefail
echo "Deploying configurations..."
# Backup current configs
BACKUP_DIR="/var/backups/configs-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$BACKUP_DIR"
# Deploy each component
rsync -av nginx/ /etc/nginx/
rsync -av ssh/ /etc/ssh/
rsync -av mysql/ /etc/mysql/
rsync -av firewall/ /etc/ufw/
# Restart services
systemctl reload nginx
systemctl reload ssh
systemctl restart mysql
echo "Deployment complete!"
echo "Backup saved to: $BACKUP_DIR"
EOF
sudo chmod +x deploy.sh
3: Infrastructure as Code Repository
# Create comprehensive IaC repository
mkdir -p ~/infrastructure
cd ~/infrastructure
git init
# Create structure
mkdir -p {ansible,terraform,docker,kubernetes,scripts,docs}
# Ansible playbooks
mkdir -p ansible/{inventory,group_vars,host_vars,roles,playbooks}
# Create .gitignore
tee .gitignore << 'EOF'
# Terraform
*.tfstate
*.tfstate.backup
.terraform/
*.tfvars
# Ansible
*.retry
vault_password
# Secrets
*.key
*.pem
*.cert
secrets/
credentials.yml
# OS
.DS_Store
Thumbs.db
*.swp
*.swo
# IDE
.vscode/
.idea/
EOF
# Create README
tee README.md << 'EOF'
# Infrastructure Repository
Infrastructure as Code (IaC) for managing server fleet.
## Components
### Ansible
- Server configuration management
- Application deployment
- Automated provisioning
### Terraform
- Cloud infrastructure provisioning
- Network configuration
- Resource management
### Docker
- Container definitions
- Docker Compose files
- Registry configurations
### Kubernetes
- Deployment manifests
- Services and ingress
- ConfigMaps and secrets
## Getting Started
```bash
# Clone repository
git clone git@github.com:company/infrastructure.git
cd infrastructure
# Run Ansible playbook
cd ansible
ansible-playbook -i inventory/production playbooks/site.yml
# Apply Terraform changes
cd terraform
terraform init
terraform plan
terraform apply
Contributing
- Create feature branch
- Make changes
- Test thoroughly
- Submit pull request EOF
Initial commit
git add . git commit -m “Initialize infrastructure repository”
### Connecting to Remote Repository
```bash
# Add remote repository
cd /etc # or your config directory
sudo git remote add origin git@github.com:company/server-configs.git
# Or use HTTPS
sudo git remote add origin https://github.com/company/server-configs.git
# Verify remote
sudo git remote -v
# Push to remote
sudo git push -u origin main
# For future pushes
sudo git push
# Pull updates from remote
sudo git pull origin main
Similarly, the Ansible Linux Automation guide demonstrates how to combine Git with configuration management tools.
What Files Should System Administrators Track with Git?
Determining which files to include in git version control balances comprehensive tracking with practical management. Consequently, focusing on meaningful configurations prevents repository bloat while maintaining complete change history.
Essential Files to Track
# System configuration
/etc/hostname
/etc/hosts
/etc/fstab
/etc/sysctl.conf
/etc/security/limits.conf
/etc/environment
/etc/locale.gen
# Network configuration
/etc/network/interfaces # Debian/Ubuntu
/etc/netplan/*.yaml # Ubuntu 18.04+
/etc/NetworkManager/ # RHEL/Fedora
# Service configurations
/etc/nginx/
/etc/apache2/ or /etc/httpd/
/etc/mysql/ or /etc/mariadb/
/etc/postgresql/
/etc/redis/
/etc/mongodb/
/etc/php/
/etc/nodejs/
# Security
/etc/ssh/sshd_config
/etc/ssh/ssh_config
/etc/pam.d/
/etc/sudoers
/etc/sudoers.d/
/etc/fail2ban/
/etc/ufw/
/etc/iptables/
/etc/selinux/config
/etc/apparmor.d/
# System services
/etc/systemd/system/
/etc/cron.d/
/etc/cron.daily/
/etc/cron.weekly/
/etc/cron.monthly/
/etc/logrotate.d/
# Monitoring and logging
/etc/rsyslog.conf
/etc/rsyslog.d/
/etc/prometheus/
/etc/grafana/
/etc/nagios/
/etc/zabbix/
# Custom scripts
/usr/local/bin/
/usr/local/sbin/
/opt/scripts/
# Application configurations
/var/www/html/config/
/opt/app/config/
Files to NEVER Track
# Create comprehensive .gitignore
cat > /etc/.gitignore << 'EOF'
# === Security Critical - NEVER TRACK ===
# User credentials
shadow
shadow-
gshadow
gshadow-
passwd-
group-
# SSL/TLS private keys
ssl/private/
*.key
*.pem
*_rsa
*_dsa
*_ecdsa
*_ed25519
# Database credentials
mysql/debian.cnf
postgresql/.pgpass
# Application secrets
*.secret
*credentials*
*_token
vault_password
# === Dynamic/Runtime Files ===
# Process IDs
*.pid
*.lock
*.sock
# Temporary files
*.tmp
*.temp
*.cache
*.log
# Mount information
mtab
fstab.d/
auto.master
# Network state
resolv.conf
network/run/
# === Package Manager Files ===
# APT
apt/sources.list.d/*.save
apt/trusted.gpg.d/
# YUM/DNF
yum.repos.d/*.rpm*
yum.repos.d/*.repo.rpmsave
# === Binary and Large Files ===
*.db
*.sqlite
*.bin
core.*
# === Backup Files ===
*~
*.bak
*.backup
*.old
*.orig
EOF
Create Tracking Template
#!/bin/bash
# /usr/local/bin/init-config-tracking.sh
# Initialize Git tracking for new server
set -euo pipefail
CONFIG_DIRS=(
"/etc/nginx"
"/etc/ssh"
"/etc/mysql"
"/etc/systemd/system"
"/usr/local/bin"
)
EXCLUDE_PATTERNS=(
"*.key"
"*.pem"
"*secret*"
"shadow*"
)
echo "Initializing configuration tracking..."
for dir in "${CONFIG_DIRS[@]}"; do
if [ -d "$dir" ]; then
echo "Tracking: $dir"
cd "$dir"
# Initialize if not already a repo
if [ ! -d ".git" ]; then
git init
# Create .gitignore
for pattern in "${EXCLUDE_PATTERNS[@]}"; do
echo "$pattern" >> .gitignore
done
# Initial commit
git add .
git commit -m "Initial snapshot of $dir - $(date -I)"
fi
fi
done
echo "Configuration tracking initialized!"
Selective Tracking Example
# .gitattributes - Control how Git handles files
# Place in repository root
# Configuration files as text
*.conf text
*.cfg text
*.ini text
*.yaml text
*.yml text
*.json text
*.xml text
# Scripts
*.sh text eol=lf
*.py text eol=lf
*.pl text eol=lf
# Diff configuration files specially
*.conf diff=ini
*.cfg diff=ini
# Don't diff binary files
*.db binary
*.sqlite binary
# Large file storage (if using Git LFS)
*.iso filter=lfs diff=lfs merge=lfs -text
*.img filter=lfs diff=lfs merge=lfs -text
The File Operations guide explains file management concepts relevant to organizing configuration repositories.
How to Use Git Branching for Safe Testing?
Git branching enables system administrators to test configuration changes safely without affecting production. Therefore, mastering branching strategies is crucial for maintaining stable infrastructure.
Understanding Branch Strategy for Infrastructure
main (production)
βββ develop (staging/integration)
β βββ feature/nginx-optimization
β βββ feature/mysql-tuning
β βββ feature/monitoring-setup
βββ release/v2.0
βββ hotfix/ssl-cert-expiry
Creating and Managing Branches
# View all branches
git branch -a
# Create new branch for testing
git checkout -b feature/nginx-optimization
# Or create without switching
git branch feature/mysql-tuning
# Switch between branches
git checkout develop
git checkout main
# Create branch from specific commit
git checkout -b hotfix/urgent-fix abc123
# Rename branch
git branch -m old-name new-name
# Delete branch
git branch -d feature/completed-task
# Force delete unmerged branch
git branch -D feature/abandoned-test
# Push branch to remote
git push -u origin feature/nginx-optimization
# Delete remote branch
git push origin --delete feature/old-branch
Safe Testing Workflow
#!/bin/bash
# Safe configuration testing workflow
BRANCH_NAME="feature/nginx-$(date +%Y%m%d)"
cd /etc/nginx
# Create test branch
sudo git checkout -b "$BRANCH_NAME"
# Make changes
sudo vi nginx.conf
# Test configuration
if sudo nginx -t; then
echo "Configuration valid"
# Commit changes
sudo git add nginx.conf
sudo git commit -m "Optimize worker processes and connections"
# Apply temporarily
sudo systemctl reload nginx
# Monitor for issues (wait 5 minutes)
echo "Monitoring for 5 minutes..."
sleep 300
# Check service status
if systemctl is-active --quiet nginx; then
echo "Service stable, merging to main"
sudo git checkout main
sudo git merge "$BRANCH_NAME"
sudo git push origin main
# Cleanup test branch
sudo git branch -d "$BRANCH_NAME"
else
echo "Service unstable, rolling back"
sudo git checkout main
sudo systemctl reload nginx
fi
else
echo "Configuration invalid, rolling back"
sudo git checkout main
fi
Branching for Different Environments
# Production branch (main)
git checkout main
# Staging branch
git checkout -b staging
# Development branch
git checkout -b development
# Feature branches from development
git checkout development
git checkout -b feature/new-vhost
# Make changes
vi sites-available/new-app.conf
git add sites-available/new-app.conf
git commit -m "Add new application vhost"
# Merge to development first
git checkout development
git merge feature/new-vhost
# Test in development
# ... testing ...
# Promote to staging
git checkout staging
git merge development
# Test in staging
# ... testing ...
# Promote to production
git checkout main
git merge staging
# Deploy to production servers
git push origin main
Branch Protection Rules
# Set up branch protection (on Git server)
# Typically done through GitHub/GitLab UI, but here's the concept:
# main branch protections:
# - Require pull request reviews
# - Require status checks to pass
# - Require branches to be up to date
# - Restrict who can push
# - Require signed commits
# Local enforcement example
cat > .git/hooks/pre-push << 'EOF'
#!/bin/bash
# Prevent direct push to main
protected_branches='main master production'
current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')
for branch in $protected_branches; do
if [ "$current_branch" = "$branch" ]; then
echo "Direct push to $branch is not allowed!"
echo "Please create a feature branch and merge via pull request."
exit 1
fi
done
exit 0
EOF
chmod +x .git/hooks/pre-push
Merge Strategies
# Fast-forward merge (default when possible)
git checkout main
git merge feature/simple-change
# Result: Linear history
# No fast-forward merge (preserve branch history)
git merge --no-ff feature/important-change
# Result: Merge commit created
# Squash merge (combine all commits)
git merge --squash feature/many-small-commits
git commit -m "Implement feature X (squashed)"
# Result: Single commit in main
# Rebase before merge (clean history)
git checkout feature/my-feature
git rebase main
git checkout main
git merge feature/my-feature
# Result: Linear history with feature commits
# Cherry-pick specific commit
git cherry-pick abc123
# Applies specific commit to current branch
The Error Handling in Bash Scripts guide demonstrates defensive programming applicable to Git automation scripts.
How to Collaborate with Teams Using Git?
Git version control excels at enabling team collaboration on infrastructure configurations. Consequently, establishing clear workflows prevents conflicts and maintains code quality.
Pull Request Workflow
# Team member 1: Create feature
git checkout -b feature/add-monitoring
vi /etc/prometheus/prometheus.yml
git add /etc/prometheus/prometheus.yml
git commit -m "Add application monitoring targets"
git push origin feature/add-monitoring
# On GitHub/GitLab: Create Pull Request
# - Add description
# - Request reviewers
# - Assign to project
# Team member 2: Review changes
git fetch origin
git checkout feature/add-monitoring
# Review files
git diff main...feature/add-monitoring
# Add review comments on specific lines
# Request changes or approve
# Team member 1: Address feedback
git checkout feature/add-monitoring
vi /etc/prometheus/prometheus.yml
git commit -am "Address review feedback: adjust scrape interval"
git push origin feature/add-monitoring
# Team lead: Merge pull request
# Via UI or command line:
git checkout main
git merge --no-ff feature/add-monitoring
git push origin main
git branch -d feature/add-monitoring
git push origin --delete feature/add-monitoring
Code Review Checklist for Infrastructure
## Configuration Review Checklist
### Syntax and Validation
- [ ] Configuration syntax is valid
- [ ] All required parameters present
- [ ] No hardcoded credentials
- [ ] File permissions appropriate
### Testing
- [ ] Tested in development environment
- [ ] Service restarts successfully
- [ ] No errors in logs
- [ ] Monitoring metrics normal
### Security
- [ ] No sensitive data exposed
- [ ] Follows security best practices
- [ ] Firewall rules reviewed
- [ ] SSL/TLS properly configured
### Documentation
- [ ] Commit message is descriptive
- [ ] README updated if needed
- [ ] Breaking changes documented
- [ ] Rollback plan defined
### Performance
- [ ] No performance degradation
- [ ] Resource limits appropriate
- [ ] Caching configured correctly
- [ ] Connection pools sized properly
Conflict Resolution
# Scenario: Two admins modified same file
# Admin 1 pushed changes first
git push origin main
# Admin 2 attempts to push
git push origin main
# ! [rejected] main -> main (fetch first)
# Admin 2 pulls changes
git pull origin main
# Auto-merging /etc/nginx/nginx.conf
# CONFLICT (content): Merge conflict in nginx.conf
# View conflicts
git status
# both modified: nginx.conf
# Open file and resolve
vi nginx.conf
# File shows:
# <<<<<<< HEAD
# worker_processes 4;
# =======
# worker_processes auto;
# >>>>>>> origin/main
# Resolve to correct value
# worker_processes auto;
# Remove conflict markers and save
# Stage resolved file
git add nginx.conf
# Complete merge
git commit -m "Merge: Resolve worker_processes conflict, using auto"
# Push merged changes
git push origin main
Conflict Prevention Strategies
# 1. Always pull before starting work
git pull origin main
# 2. Communicate with team
# Use Slack/Teams to coordinate changes
# 3. Use branch protection
# Require pull requests for main
# 4. Smaller, frequent commits
git add specific-file.conf
git commit -m "Update specific setting"
# 5. Rebase your changes
git fetch origin
git rebase origin/main
# Applies your changes on top of latest
# 6. Use lock files for critical changes
#!/bin/bash
LOCKFILE="/tmp/config-edit.lock"
if ! mkdir "$LOCKFILE" 2>/dev/null; then
echo "Someone else is editing configs. Try later."
exit 1
fi
trap "rmdir $LOCKFILE" EXIT
# Make changes
vi /etc/nginx/nginx.conf
Team Communication Tools
# Git commit message template with team context
cat > ~/.gitmessage << 'EOF'
# [TYPE] Brief description (50 chars)
#
# More detailed explanation (wrap at 72 chars)
#
# Motivation for change:
# - Problem being solved
# - Why this approach
#
# Testing performed:
# - Unit tests: pass/fail
# - Integration tests: pass/fail
# - Manual testing: describe
#
# Deployment notes:
# - Service restart required: yes/no
# - Breaking changes: describe
# - Rollback plan: describe
#
# Related:
# - Ticket: JIRA-123
# - Related PR: #456
# - Docs: https://wiki.company.com/page
#
# Reviewers: @admin1 @admin2
EOF
git config commit.template ~/.gitmessage
# Example commit following template
git commit
# Editor opens with template filled in
Remote Repository Management
# View remotes
git remote -v
# Add multiple remotes (backup strategy)
git remote add github git@github.com:company/infrastructure.git
git remote add gitlab git@gitlab.company.local:ops/infrastructure.git
git remote add backup git@backup-server:/git/infrastructure.git
# Push to multiple remotes
git push github main
git push gitlab main
git push backup main
# Or set up to push to all
git remote set-url --add --push origin git@github.com:company/infrastructure.git
git remote set-url --add --push origin git@gitlab.company.local:ops/infrastructure.git
git push origin main # Pushes to both
# Fetch from specific remote
git fetch github
# Track remote branch
git checkout -b feature/new --track origin/feature/new
# View remote branches
git branch -r
# Prune deleted remote branches
git remote prune origin
Additionally, the Introduction to Ansible guide shows how teams collaborate on infrastructure automation.
What Are Git Hooks and How to Automate Workflows?
Git hooks enable automated git version control workflows, ensuring consistency and preventing errors. Therefore, implementing hooks streamlines infrastructure management and enforces policies.
Understanding Git Hooks
# Hook locations
.git/hooks/
# Available hooks
ls -la .git/hooks/
# applypatch-msg.sample
# commit-msg.sample
# pre-commit.sample
# pre-push.sample
# pre-rebase.sample
# prepare-commit-msg.sample
# update.sample
# Enable hook (remove .sample)
mv .git/hooks/pre-commit.sample .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
Pre-Commit Hook: Validate Configurations
#!/bin/bash
# .git/hooks/pre-commit
# Validate configurations before commit
echo "Running pre-commit checks..."
# Check Nginx configuration
if git diff --cached --name-only | grep -q "nginx.*\.conf"; then
echo "Checking Nginx configuration..."
for file in $(git diff --cached --name-only | grep "nginx.*\.conf"); do
if [ -f "$file" ]; then
if ! nginx -t -c "$file" 2>/dev/null; then
echo "ERROR: Invalid Nginx configuration in $file"
exit 1
fi
fi
done
echo "β Nginx configuration valid"
fi
# Check Apache configuration
if git diff --cached --name-only | grep -q "apache2.*\.conf\|httpd.*\.conf"; then
echo "Checking Apache configuration..."
if ! apachectl configtest 2>/dev/null; then
echo "ERROR: Invalid Apache configuration"
exit 1
fi
echo "β Apache configuration valid"
fi
# Check systemd unit files
if git diff --cached --name-only | grep -q "\.service$\|\.timer$"; then
echo "Checking systemd units..."
for file in $(git diff --cached --name-only | grep "\.service$\|\.timer$"); do
if [ -f "$file" ]; then
if ! systemd-analyze verify "$file" 2>/dev/null; then
echo "WARNING: Systemd unit validation failed for $file"
fi
fi
done
echo "β Systemd units checked"
fi
# Check for secrets
echo "Scanning for secrets..."
if git diff --cached | grep -iE '(password|secret|token|api_key|private_key).*=.*[^*]'; then
echo "ERROR: Possible secret detected in commit"
echo "Please remove sensitive data before committing"
exit 1
fi
echo "β No secrets detected"
# Check YAML syntax
if git diff --cached --name-only | grep -q "\.ya\?ml$"; then
echo "Checking YAML syntax..."
for file in $(git diff --cached --name-only | grep "\.ya\?ml$"); do
if [ -f "$file" ]; then
if ! python3 -c "import yaml; yaml.safe_load(open('$file'))" 2>/dev/null; then
echo "ERROR: Invalid YAML syntax in $file"
exit 1
fi
fi
done
echo "β YAML syntax valid"
fi
echo "All pre-commit checks passed!"
exit 0
Post-Commit Hook: Automated Deployment
#!/bin/bash
# .git/hooks/post-commit
# Automatically deploy to staging after commit
BRANCH=$(git symbolic-ref --short HEAD)
STAGING_SERVER="staging.company.local"
if [ "$BRANCH" = "develop" ]; then
echo "Deploying to staging server..."
# Push to remote
git push origin develop
# SSH to staging and pull changes
ssh deploy@$STAGING_SERVER << 'ENDSSH'
cd /opt/infrastructure
git pull origin develop
sudo systemctl reload nginx
sudo systemctl reload apache2
echo "Staging deployment complete"
ENDSSH
echo "Deployment to staging complete!"
fi
Pre-Push Hook: Prevent Direct Push to Main
#!/bin/bash
# .git/hooks/pre-push
# Prevent direct pushes to protected branches
protected_branches=('main' 'master' 'production')
current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')
for protected in "${protected_branches[@]}"; do
if [ "$protected" = "$current_branch" ]; then
echo "Direct push to $protected branch is prohibited!"
echo "Please create a feature branch and submit a pull request."
echo ""
echo "To create a feature branch:"
echo " git checkout -b feature/my-changes"
echo " git push origin feature/my-changes"
exit 1
fi
done
echo "Push allowed to $current_branch branch"
exit 0
Commit Message Validation Hook
#!/bin/bash
# .git/hooks/commit-msg
# Enforce commit message format
commit_msg_file=$1
commit_msg=$(cat "$commit_msg_file")
# Pattern: [TYPE] Message
# Types: CONFIG, SCRIPT, SECURITY, UPDATE, FIX, FEATURE
pattern='^\[(CONFIG|SCRIPT|SECURITY|UPDATE|FIX|FEATURE)\] .{10,}$'
if ! echo "$commit_msg" | grep -qE "$pattern"; then
echo "ERROR: Invalid commit message format"
echo ""
echo "Commit message must follow this format:"
echo " [TYPE] Brief description (at least 10 characters)"
echo ""
echo "Valid types: CONFIG, SCRIPT, SECURITY, UPDATE, FIX, FEATURE"
echo ""
echo "Examples:"
echo " [CONFIG] Update Nginx worker processes to 4"
echo " [SECURITY] Enable fail2ban for SSH protection"
echo " [SCRIPT] Add automated backup script"
exit 1
fi
exit 0
Server-Side Hooks (Bare Repository)
# On Git server: /srv/git/infrastructure.git/hooks/update
#!/bin/bash
# Server-side update hook
# Enforce policies on push
refname="$1"
oldrev="$2"
newrev="$3"
# Only allow fast-forward merges to main
if [ "$refname" = "refs/heads/main" ]; then
# Check if fast-forward
if ! git merge-base --is-ancestor "$oldrev" "$newrev"; then
echo "Non-fast-forward push to main rejected"
echo "Please rebase your changes"
exit 1
fi
fi
# Require signed commits on main
if [ "$refname" = "refs/heads/main" ]; then
if ! git verify-commit "$newrev" 2>/dev/null; then
echo "Unsigned commit pushed to main rejected"
echo "Please sign your commits: git commit -S"
exit 1
fi
fi
exit 0
Post-Receive Hook: Automated Notifications
#!/bin/bash
# Server-side post-receive hook
# Send notifications after push
while read oldrev newrev refname; do
branch=$(echo "$refname" | sed -n 's/^refs\/heads\///p')
if [ -n "$branch" ]; then
# Get commit details
author=$(git log -1 --format='%an' "$newrev")
message=$(git log -1 --format='%s' "$newrev")
# Send Slack notification
curl -X POST -H 'Content-type: application/json' \
--data "{
\"text\": \"New push to *$branch* by $author\",
\"attachments\": [{
\"text\": \"$message\",
\"color\": \"good\"
}]
}" \
"$SLACK_WEBHOOK_URL"
# Send email notification
echo "Branch: $branch
Author: $author
Commit: $message
View: https://git.company.local/infrastructure/commit/$newrev" | \
mail -s "Git Push: $branch" ops-team@company.com
fi
done
The Cron and Task Scheduling guide demonstrates automation concepts applicable to Git hook workflows.
How to Implement Infrastructure as Code with Git?
Git version control forms the foundation of Infrastructure as Code (IaC), enabling version-controlled, reproducible infrastructure deployments. Consequently, combining Git with IaC tools creates auditable, automated infrastructure management.
Complete IaC Repository Structure
# Create comprehensive IaC repository
mkdir -p ~/infrastructure-as-code
cd ~/infrastructure-as-code
# Initialize Git
git init
# Create organized structure
mkdir -p {
ansible/{inventory,group_vars,host_vars,roles,playbooks},
terraform/{environments,modules},
docker/{services,compose},
kubernetes/{base,overlays,secrets},
scripts/{deployment,maintenance,monitoring},
docs/{runbooks,architecture,procedures}
}
# Create README
cat > README.md << 'EOF'
# Infrastructure as Code Repository
Complete infrastructure management using Git version control.
## Repository Structure
βββ ansible/ # Configuration management β βββ inventory/ # Host definitions β βββ group_vars/ # Group variables β βββ roles/ # Reusable roles β βββ playbooks/ # Automation playbooks βββ terraform/ # Cloud infrastructure β βββ environments/ # Dev, staging, prod β βββ modules/ # Reusable modules βββ docker/ # Container definitions βββ kubernetes/ # K8s manifests βββ scripts/ # Automation scripts βββ docs/ # Documentation
## Workflow
1. Create feature branch
2. Make infrastructure changes
3. Test in development
4. Submit pull request
5. Review and merge
6. Automated deployment
EOF
git add .
git commit -m "[INIT] Initialize IaC repository structure"
Ansible Integration with Git
# ansible/playbooks/deploy-from-git.yml
---
- name: Deploy configurations from Git
hosts: all
become: yes
vars:
git_repo: "git@github.com:company/infrastructure.git"
git_branch: "main"
deploy_path: "/opt/infrastructure"
tasks:
- name: Install Git
apt:
name: git
state: present
- name: Clone or update repository
git:
repo: "{{ git_repo }}"
dest: "{{ deploy_path }}"
version: "{{ git_branch }}"
force: yes
register: git_result
- name: Show Git changes
debug:
msg: "Repository updated: {{ git_result.changed }}"
- name: Deploy Nginx configuration
copy:
src: "{{ deploy_path }}/configs/nginx/"
dest: /etc/nginx/
remote_src: yes
notify: Reload Nginx
when: git_result.changed
- name: Deploy scripts
copy:
src: "{{ deploy_path }}/scripts/"
dest: /usr/local/bin/
mode: '0755'
remote_src: yes
when: git_result.changed
handlers:
- name: Reload Nginx
service:
name: nginx
state: reloaded
Terraform with Git Workflow
# terraform/environments/production/main.tf
terraform {
required_version = ">= 1.0"
backend "s3" {
bucket = "company-terraform-state"
key = "production/terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "terraform-locks"
}
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = var.aws_region
}
# Track all Terraform files in Git
module "network" {
source = "../../modules/network"
environment = "production"
vpc_cidr = var.vpc_cidr
}
module "compute" {
source = "../../modules/compute"
environment = "production"
instance_count = var.instance_count
}
# Terraform Git workflow script
#!/bin/bash
# scripts/terraform-deploy.sh
set -euo pipefail
ENVIRONMENT=$1
ACTION=${2:-plan}
cd "terraform/environments/$ENVIRONMENT"
# Ensure Git is clean
if ! git diff-index --quiet HEAD --; then
echo "ERROR: Uncommitted changes detected"
echo "Please commit or stash changes before deployment"
exit 1
fi
# Pull latest changes
git pull origin main
# Initialize Terraform
terraform init
# Validate configuration
terraform validate
# Plan changes
terraform plan -out=tfplan
# If apply requested, proceed
if [ "$ACTION" = "apply" ]; then
# Show plan
terraform show tfplan
# Confirm
read -p "Apply these changes? (yes/no): " confirm
if [ "$confirm" = "yes" ]; then
# Apply
terraform apply tfplan
# Commit state file reference
git add terraform.tfstate.backup
git commit -m "[TERRAFORM] Applied changes to $ENVIRONMENT"
git push origin main
fi
fi
# Cleanup
rm -f tfplan
Docker Compose with Git
# docker/compose/production.yml
version: '3.8'
services:
web:
image: nginx:latest
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/sites:/etc/nginx/sites-available:ro
environment:
- NGINX_HOST=example.com
deploy:
replicas: 3
update_config:
parallelism: 1
delay: 10s
app:
build:
context: ../../app
dockerfile: Dockerfile
environment:
- DB_HOST=postgres
- DB_PORT=5432
depends_on:
- postgres
- redis
postgres:
image: postgres:14
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
secrets:
- db_password
volumes:
postgres_data:
secrets:
db_password:
external: true
# Deployment script with Git integration
#!/bin/bash
# scripts/docker-deploy.sh
ENVIRONMENT=$1
COMPOSE_FILE="docker/compose/${ENVIRONMENT}.yml"
# Pull latest changes
git pull origin main
# Verify compose file exists
if [ ! -f "$COMPOSE_FILE" ]; then
echo "ERROR: Compose file not found: $COMPOSE_FILE"
exit 1
fi
# Validate compose file
docker-compose -f "$COMPOSE_FILE" config > /dev/null
# Deploy
docker-compose -f "$COMPOSE_FILE" up -d
# Log deployment
git log -1 --format="%H %s" >> deployment-log.txt
git add deployment-log.txt
git commit -m "[DEPLOY] Deployed $ENVIRONMENT $(date -I)"
git push origin main
Kubernetes Manifests in Git
# kubernetes/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: company/webapp:latest
ports:
- containerPort: 8080
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
# GitOps deployment script
#!/bin/bash
# scripts/k8s-deploy.sh
NAMESPACE=$1
MANIFEST_DIR="kubernetes/overlays/$NAMESPACE"
# Ensure clean Git state
if ! git diff-index --quiet HEAD --; then
echo "ERROR: Uncommitted changes"
exit 1
fi
# Pull latest
git pull origin main
# Apply manifests
kubectl apply -k "$MANIFEST_DIR"
# Wait for rollout
kubectl rollout status deployment/web-app -n "$NAMESPACE"
# Tag release
VERSION="v$(date +%Y%m%d-%H%M%S)"
git tag -a "$VERSION" -m "Deployment to $NAMESPACE"
git push origin "$VERSION"
echo "Deployment complete: $VERSION"
The Docker Fundamentals guide explains containerization concepts that integrate with Git-based workflows.
Best Practices for Git Version Control in Production
Professional git version control implementation requires adherence to established patterns ensuring reliability, security, and team collaboration. Therefore, following these practices prevents common pitfalls and maintains code quality.
1. Commit Message Standards
# Good commit messages
[CONFIG] Update Nginx worker processes from 2 to 4
[SECURITY] Enable fail2ban for SSH brute force protection
[SCRIPT] Add automated MySQL backup with compression
[FIX] Resolve Apache virtual host configuration syntax error
[UPDATE] Upgrade PHP to 8.1 for security patches
# Bad commit messages
"fix"
"update config"
"changes"
"asdf"
"fixed bug"
Commit Message Template:
# Configure commit template
cat > ~/.gitmessage << 'EOF'
[TYPE] Brief description (max 50 characters)
Why: Reason for this change
- Problem being solved
- Impact on system
How: Implementation details
- Key changes made
- Services affected
Testing:
- Configuration validated: yes/no
- Service restart successful: yes/no
- Monitoring checks: normal/abnormal
Rollback:
- Revert command: git revert <commit>
- Impact: describe
Related: TICKET-123, PR#456
EOF
git config --global commit.template ~/.gitmessage
2. Branching Naming Conventions
# Feature branches
feature/add-prometheus-monitoring
feature/nginx-optimization
feature/ssl-automation
# Bug fix branches
fix/apache-memory-leak
fix/mysql-connection-timeout
fix/ssl-cert-renewal
# Hotfix branches (production emergencies)
hotfix/security-patch-log4j
hotfix/disk-full-cleanup
hotfix/nginx-crash-recovery
# Release branches
release/v2.0
release/v2.1-staging
# Environment branches
main # Production
staging # Staging environment
development # Development environment
3. .gitignore Best Practices
# Comprehensive .gitignore for infrastructure
# === Secrets and Credentials ===
*.key
*.pem
*.cert
*.crt
*.p12
*_rsa
*_dsa
*_ecdsa
.env
.env.local
*secret*
*password*
*credential*
vault_password
.vault-pass
# === Terraform ===
*.tfstate
*.tfstate.backup
*.tfvars
.terraform/
.terraform.lock.hcl
crash.log
# === Ansible ===
*.retry
vault_pass.txt
.ansible/
# === Docker ===
.dockerenv
docker-compose.override.yml
# === Kubernetes ===
*-secret.yaml
kubeconfig*
# === IDE and Editors ===
.vscode/
.idea/
*.swp
*.swo
*~
# === OS ===
.DS_Store
Thumbs.db
desktop.ini
# === Logs ===
*.log
logs/
npm-debug.log*
# === Temporary ===
*.tmp
*.temp
*.bak
*.backup
*.old
tmp/
temp/
# === Runtime ===
*.pid
*.sock
*.lock
4. Security Best Practices
# Enable commit signing
gpg --full-generate-key
gpg --list-secret-keys --keyid-format LONG
# Copy key ID
git config --global user.signingkey <KEY_ID>
git config --global commit.gpgsign true
# Sign commits
git commit -S -m "[CONFIG] Update security settings"
# Verify signatures
git log --show-signature
# Prevent unencrypted secrets
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/bash
if git diff --cached | grep -i "password.*=" | grep -v "password.*=\*\*\*"; then
echo "ERROR: Unencrypted password detected!"
exit 1
fi
EOF
chmod +x .git/hooks/pre-commit
# Use git-secrets tool
git secrets --install
git secrets --register-aws
5. Repository Organization
# Monorepo structure
infrastructure/
βββ README.md
βββ .gitignore
βββ .gitattributes
βββ ansible/
β βββ README.md
β βββ inventory/
β βββ playbooks/
β βββ roles/
βββ terraform/
β βββ README.md
β βββ modules/
β βββ environments/
βββ kubernetes/
β βββ README.md
β βββ base/
β βββ overlays/
βββ docker/
β βββ README.md
β βββ services/
βββ scripts/
β βββ README.md
β βββ deployment/
β βββ backup/
β βββ monitoring/
βββ docs/
β βββ architecture/
β βββ runbooks/
β βββ procedures/
βββ tests/
βββ integration/
βββ validation/
# Each directory contains:
# - README.md (purpose, usage)
# - CONTRIBUTING.md (how to contribute)
# - CHANGELOG.md (version history)
6. Backup and Disaster Recovery
#!/bin/bash
# /usr/local/bin/git-backup.sh
# Automated Git repository backup
BACKUP_DIR="/backup/git"
REPOS=(
"/etc/.git"
"/opt/infrastructure"
"/srv/git/configs.git"
)
mkdir -p "$BACKUP_DIR"
for repo in "${REPOS[@]}"; do
if [ -d "$repo" ]; then
REPO_NAME=$(basename "$repo" .git)
BACKUP_FILE="$BACKUP_DIR/${REPO_NAME}-$(date +%Y%m%d).bundle"
cd "$repo"
# Create bundle (complete repository backup)
git bundle create "$BACKUP_FILE" --all
# Verify bundle
if git bundle verify "$BACKUP_FILE" > /dev/null 2>&1; then
echo "β Backup created: $BACKUP_FILE"
# Upload to S3
aws s3 cp "$BACKUP_FILE" \
s3://company-backups/git/"$(basename $BACKUP_FILE)"
fi
# Keep only last 30 days
find "$BACKUP_DIR" -name "${REPO_NAME}-*.bundle" \
-mtime +30 -delete
fi
done
7. Continuous Integration/Deployment
# .github/workflows/infrastructure.yml
name: Infrastructure CI/CD
on:
push:
branches: [ main, staging ]
pull_request:
branches: [ main ]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Validate Ansible
run: |
pip install ansible-lint
ansible-lint ansible/playbooks/
- name: Validate Terraform
run: |
terraform fmt -check -recursive terraform/
terraform validate terraform/environments/production/
- name: Validate Kubernetes
run: |
kubectl --dry-run=client apply -k kubernetes/base/
- name: Scan for secrets
uses: trufflesecurity/trufflehog@main
with:
path: ./
deploy-staging:
needs: validate
if: github.ref == 'refs/heads/staging'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Deploy to staging
run: |
ansible-playbook -i inventory/staging \
ansible/playbooks/site.yml
deploy-production:
needs: validate
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: production
steps:
- uses: actions/checkout@v2
- name: Deploy to production
run: |
ansible-playbook -i inventory/production \
ansible/playbooks/site.yml
- name: Create release
uses: actions/create-release@v1
with:
tag_name: v${{ github.run_number }}
release_name: Release ${{ github.run_number }}
8. Documentation Standards
# README.md Template
## Purpose
Brief description of what this repository manages
## Prerequisites
- Git 2.30+
- Ansible 2.10+
- Terraform 1.0+
- Access credentials
## Quick Start
```bash
# Clone repository
git clone git@github.com:company/infrastructure.git
cd infrastructure
# Setup
./scripts/setup.sh
# Deploy to staging
./scripts/deploy.sh staging
Repository Structure
Explain directory organization
Workflows
- Feature development process
- Pull request guidelines
- Deployment procedures
- Rollback procedures
Configuration
How to configure for different environments
Troubleshooting
Common issues and solutions
Contributing
How to contribute changes
Contacts
- Team lead: name@company.com
- On-call: oncall@company.com
- Slack: #infrastructure
The [Git Documentation](https://git-scm.com/doc) provides comprehensive reference information for all Git commands and workflows.
---
## Frequently Asked Questions {#faq}
### Should I track my entire /etc directory?
**Recommendation:** Track selectively, not everything.
```bash
# Good approach: Track specific directories
cd /etc
sudo git init
sudo git add nginx/ mysql/ ssh/ systemd/
# Add .gitignore for sensitive files
sudo git add .gitignore
sudo git commit -m "Initial selective /etc tracking"
# Bad approach: Track everything
cd /etc
sudo git init
sudo git add .
# Includes secrets, temporary files, and system-generated content
How do I handle merge conflicts in configuration files?
# When conflict occurs
git status
# both modified: nginx.conf
# View conflict
cat nginx.conf
# <<<<<<< HEAD
# worker_processes 4;
# =======
# worker_processes auto;
# >>>>>>> feature/optimization
# Options:
# 1. Keep yours: git checkout --ours nginx.conf
# 2. Keep theirs: git checkout --theirs nginx.conf
# 3. Manual resolution: edit file, remove markers
# After resolving
git add nginx.conf
git commit -m "[MERGE] Resolve nginx worker_processes conflict"
How often should I commit configuration changes?
Best Practice: Commit atomically – one logical change per commit.
# Good: Atomic commits
git commit -m "[CONFIG] Update Nginx worker processes"
git commit -m "[SECURITY] Enable SSL for vhost"
git commit -m "[FIX] Correct MySQL connection timeout"
# Bad: Bundled unrelated changes
git commit -m "Various updates to configs"
Can I use Git for binary files like databases?
Not recommended for large binary files. Use Git LFS or alternative solutions:
# Install Git LFS
sudo apt install git-lfs
git lfs install
# Track binary files
git lfs track "*.db"
git lfs track "*.sqlite"
# Or use dedicated backup solutions
# - pg_dump for PostgreSQL
# - mysqldump for MySQL
# - mongodump for MongoDB
How do I recover from accidental force push?
# View reflog (local history)
git reflog
# Find commit before force push
git log --graph --oneline --all
# Reset to previous state
git reset --hard abc123
# Force push correction (if you have access)
git push --force origin main
# If remote is damaged, restore from backup bundle
git clone /backup/repository.bundle recovered-repo
Should each server have its own repository?
Two approaches:
# Approach 1: Single repository for all servers
infrastructure/
βββ inventory/
β βββ server1/
β βββ server2/
β βββ server3/
βββ ansible/
βββ playbooks/
# Approach 2: Per-server repositories
server1-config/
server2-config/
server3-config/
# Recommendation: Single repo with branches per environment
βββ main (production)
βββ staging
βββ development
How do I automate Git operations?
#!/bin/bash
# Automated configuration backup
cd /etc
# Check for changes
if ! git diff-index --quiet HEAD --; then
# Commit changes
git add -A
git commit -m "Automated backup $(date -I)"
git push origin main
# Send notification
echo "Configuration changes backed up" | \
mail -s "Git Backup Success" admin@company.com
fi
Troubleshooting Common Git Issues
Repository Too Large
Git operations are slow, repository size growing
# Diagnose
git count-objects -vH
# Find large files
git rev-list --objects --all | \
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \
awk '/^blob/ {print substr($0,6)}' | \
sort --numeric-sort --key=2 | \
tail -n 10
# Remove large files from history
git filter-branch --tree-filter 'rm -f path/to/large/file' HEAD
# Or use BFG Repo-Cleaner (faster)
java -jar bfg.jar --strip-blobs-bigger-than 10M my-repo.git
# Force push cleaned repository
git push origin --force --all
Detached HEAD State
Not on any branch
# Check current state
git status
# HEAD detached at abc123
# Solution 1: Create new branch from current position
git checkout -b recovery-branch
# Solution 2: Return to previous branch
git checkout main
# Solution 3: Discard changes and return
git checkout main
git reset --hard origin/main
Accidental Commit to Wrong Branch
Committed to main instead of feature branch
# Don't panic! Fix with these steps:
# 1. Create new branch from current position
git branch feature/my-changes
# 2. Reset main to before the commit
git reset --hard HEAD~1
# 3. Switch to new branch
git checkout feature/my-changes
# 4. Push feature branch
git push origin feature/my-changes
# Alternative: If already pushed to main
git revert HEAD
git push origin main
Git Refusing to Overwrite Local Changes
“error: Your local changes would be overwritten”
# Option 1: Stash changes
git stash
git pull origin main
git stash pop
# Option 2: Commit changes first
git add -A
git commit -m "WIP: save current work"
git pull origin main
# Option 3: Discard local changes (careful!)
git reset --hard HEAD
git pull origin main
# Option 4: Keep local changes, but update
git fetch origin
git rebase origin/main
Cannot Push Due to Non-Fast-Forward
“Updates were rejected because the tip of your current branch is behind”
# Diagnose
git fetch origin
git log HEAD..origin/main --oneline
# Solution 1: Pull and merge
git pull origin main
# Resolve any conflicts
git push origin main
# Solution 2: Rebase (cleaner history)
git fetch origin
git rebase origin/main
git push origin main
# Solution 3: Force push (dangerous!)
# Only if you're sure no one else is using the branch
git push --force-with-lease origin main
Merge Conflicts
Conflicts during merge or rebase
# View conflicted files
git status
# For each conflict, choose strategy:
# Option 1: Accept yours
git checkout --ours conflicted-file.conf
git add conflicted-file.conf
# Option 2: Accept theirs
git checkout --theirs conflicted-file.conf
git add conflicted-file.conf
# Option 3: Merge tool
git mergetool
# Option 4: Manual resolution
vim conflicted-file.conf
# Edit file, remove conflict markers
git add conflicted-file.conf
# Complete merge
git commit -m "[MERGE] Resolve conflicts in configuration"
# If things go wrong, abort
git merge --abort # or
git rebase --abort
Sensitive Data Committed
Password or key committed to repository
# IMMEDIATE ACTION:
# 1. Change the compromised credential
# 2. Remove from Git history
# Remove from last commit
git reset HEAD~1
# Edit file to remove sensitive data
git add file
git commit -m "Remove sensitive data"
# Remove from history (all commits)
git filter-branch --tree-filter \
'sed -i "s/password=SECRET/password=REMOVED/g" config.yml' HEAD
# Or use BFG (recommended)
echo "password=SECRET" > passwords.txt
bfg --replace-text passwords.txt my-repo.git
# Force push
git push origin --force --all
# Notify team to re-clone repository
Git Hooks Not Executing
Pre-commit hook not running
# Check hook exists and is executable
ls -la .git/hooks/pre-commit
# Should show: -rwxr-xr-x
# Make executable
chmod +x .git/hooks/pre-commit
# Test hook directly
.git/hooks/pre-commit
# Verify shell interpreter
head -n 1 .git/hooks/pre-commit
# Should show: #!/bin/bash or #!/bin/sh
# Debug hook execution
bash -x .git/hooks/pre-commit
The Linux Command Line Tools guide provides comprehensive information about shell commands used in Git workflows.
Real-World Use Cases
1: Multi-Server Configuration Management
#!/bin/bash
# Manage configurations across 100+ servers
SERVERS_FILE="inventory/production-servers.txt"
CONFIG_REPO="/opt/infrastructure"
# Pull latest configurations
cd "$CONFIG_REPO"
git pull origin main
# Deploy to each server
while read server; do
echo "Deploying to $server..."
# SSH and update
ssh -T "$server" << 'ENDSSH'
cd /etc
sudo git pull origin main
sudo systemctl reload nginx
sudo systemctl reload apache2
ENDSSH
if [ $? -eq 0 ]; then
echo "β $server updated successfully"
# Log successful deployment
git log -1 --format="%H" >> "deployments/$server-$(date -I).log"
else
echo "β $server update failed"
# Alert on failure
echo "Deployment to $server failed" | \
mail -s "Deployment Alert" ops@company.com
fi
done < "$SERVERS_FILE"
# Commit deployment log
git add deployments/
git commit -m "[DEPLOY] Automated deployment $(date -I)"
git push origin main
2: Disaster Recovery
#!/bin/bash
# Complete infrastructure recovery from Git
BACKUP_BUNDLE="/backup/infrastructure-20251008.bundle"
RECOVERY_DIR="/opt/infrastructure-recovery"
echo "Starting disaster recovery..."
# Clone from bundle
git clone "$BACKUP_BUNDLE" "$RECOVERY_DIR"
cd "$RECOVERY_DIR"
# Verify repository integrity
if ! git fsck --full; then
echo "ERROR: Repository corruption detected"
exit 1
fi
# Deploy configurations
ansible-playbook -i inventory/production playbooks/site.yml
# Restore databases
./scripts/restore-databases.sh
# Restore certificates
./scripts/restore-certificates.sh
# Verify all services
./scripts/verify-services.sh
echo "Disaster recovery complete!"
echo "Total recovery time: $(date)"
3: Compliance Auditing
#!/bin/bash
# Generate compliance audit report from Git
REPORT_FILE="compliance-audit-$(date -I).txt"
cat > "$REPORT_FILE" << 'EOF'
# Infrastructure Compliance Audit Report
Generated: $(date)
## Configuration Changes (Last 90 Days)
### Critical Security Changes
EOF
# Find security-related commits
git log --since="90 days ago" --grep="SECURITY" \
--pretty=format:"%ad - %an - %s" --date=short >> "$REPORT_FILE"
echo -e "\n\n### Firewall Rule Changes" >> "$REPORT_FILE"
git log --since="90 days ago" -- "*/firewall/*" \
--pretty=format:"%ad - %an - %s" --date=short >> "$REPORT_FILE"
echo -e "\n\n### SSH Configuration Changes" >> "$REPORT_FILE"
git log --since="90 days ago" -- "*/ssh/*" \
--pretty=format:"%ad - %an - %s" --date=short >> "$REPORT_FILE"
echo -e "\n\n### Unauthorized Changes Detection" >> "$REPORT_FILE"
# Check for commits outside business hours
git log --since="90 days ago" --all \
--pretty=format:"%ad - %an - %s" --date=format:"%H" | \
awk '$1 < 8 || $1 > 18' >> "$REPORT_FILE"
# Send report
mail -s "Compliance Audit Report" compliance@company.com < "$REPORT_FILE"
Additional Resources
Official Documentation
- Git Official Documentation – Complete Git reference
- Pro Git Book – Comprehensive free book
- Git Reference Manual – Command reference
- GitHub Guides – GitHub-specific tutorials
Infrastructure as Code Resources
- Terraform Documentation – IaC with Terraform
- Ansible Documentation – Configuration management
- Kubernetes Documentation – Container orchestration
Related LinuxTips.pro Guides
- Ansible Linux Automation – Configuration management
- Bash Scripting Basics – Automation scripts
- Linux Task Scheduling – Automated workflows
- SSH Server Security – Secure access
- Linux Security Essentials – Security hardening
Git Tools and Utilities
- GitKraken – Visual Git client
- Git LFS – Large file storage
- BFG Repo-Cleaner – Repository cleanup
- git-secrets – Prevent committing secrets
Conclusion
Mastering git version control transforms system administration from reactive firefighting to proactive infrastructure management. By tracking every configuration change, enabling safe testing through branching, facilitating team collaboration, and providing instant rollback capabilities, Git becomes the foundation of reliable infrastructure operations.
Furthermore, combining Git with Infrastructure as Code tools like Ansible, Terraform, and Kubernetes creates reproducible, version-controlled environments that scale from single servers to global infrastructure. The comprehensive audit trail Git provides meets compliance requirements while the distributed nature ensures configurations are never lost.
Remember to start with simple tracking of critical configurations, gradually expanding coverage as you become comfortable with Git workflows. Implement hooks for automation, establish clear branching strategies for your team, and integrate Git into your deployment pipelines. Regular commits with descriptive messages create invaluable documentation of infrastructure evolution.
Start implementing git version control for your infrastructure today, and you’ll immediately gain visibility, control, and confidence in managing your Linux systems, whether you’re responsible for one server or thousands.
Last Updated: October 2025