Rsync Backup Tool

Rsync Backup Tool pulls backups from many servers in one run: folders common to all hosts plus host-specific ones, each run into its own timestamped folder, all driven by one YAML file.

Source on GitHub · Summary on chaintools.tech · MIT licence

Requirements

• Python 3.7 or newer
• rsync and an SSH client on the backup machine, rsync on every host: sudo apt install -y rsync openssh-client
• Key-based SSH access from the backup machine to every host
• On every host, the SSH user must be able to run sudo without a password (at least for test and rsync), because folders are checked and copied with sudo

Install

git clone https://github.com/qf3l3k/rsync_backup_tool.git && cd rsync_backup_tool
python3 -m venv .venv && . .venv/bin/activate
pip install .

Configuration

destination: /srv/backups
log_directory: /srv/backups/.logs
log_file: rsync_backup_tool.log
log_level: INFO

common_folders:
  - path: /etc
    pull_entire_folder: true
  - path: /home/validator/.bitsongd/config
    pull_entire_folder: false

hosts:
  backup@node-1.example.com:
    specific_folders:
      - path: /opt/app
        pull_entire_folder: true
  node-2:          # an alias from ~/.ssh/config works too
    specific_folders: []

Key	Meaning
destination	Base folder for backups (required)
log_directory	Folder for logs — set it explicitly, see below
log_file	Main log file name (default rsync_backup_tool.log)
log_level	DEBUG, INFO, … (required)
common_folders	Folders backed up from every host (required)
hosts	Map of SSH destinations to specific_folders (required)
path	Folder on the host
pull_entire_folder	true: the folder with all subfolders; false: only the files directly in it

Each key under hosts is used as-is as the SSH destination, so it can be user@host, a hostname or an alias from ~/.ssh/config.

Usage

rsync_backup_tool --config config.yaml

Each run creates <destination>/<YYYYMMDDHHMM>/ and copies every folder to <destination>/<YYYYMMDDHHMM>/<host><path>. Folders that don't exist on a host are skipped with a warning. Run it from cron for regular backups.

Logs

• The main log (log_file in log_directory) is rewritten on every run.
• rsync output is appended to rsync.log in log_directory. Set log_directory in the configuration: without it, the main log goes to .logs/ but rsync.log to logs/, which isn't created automatically.

TIP

The backed-up /etc/ufw folders can be read by UFW Parser in local mode to review the firewall rules of all hosts at once.