• Features
• Setup
• Command line usage
• Backup repository Config
• Host config
• Host config : schedule format
• Workflow
• TODO
• Credits

Features

• per host backup
• per group backup
• per features (database | files) backups
• schedule backup with automatic cron generation
• simple configuration via yaml file
• command line utility
• retention period
• backup reports with success / failed

Setup

1. make and activate a virtualenv
2. pip install git+https://git.tng-concepts.com/TNG/central-backup

Sudo password will be ask to

• copy central_backup utility in /usr/local/bin.
• create /etc/central-backup directories
• create a default config files

3. create your hosts to backup under /etc/central-backup/conf.d (see below)
4. fill /etc/central-backup/config.yml
5. Run central_backup fix (fix permissions) and central_backup schedule (crontab creation)

You got it !

Command line usage

Python utility to manage centralized backups with ssh.

Usage: 
  central_backup (backup | restore | fix | schedule) [options]
  central_backup (backup | restore) (--databases=<STR> --hosts=<STR>) [--features=<STR>]
  central_backup (backup | restore) [--hosts=<STR>] [--features=<STR>]

Commands:
    backup  Launch backup action.
    restore Launch restore action
    schedule Crontab creation regarding schedule information in host files.
    fix     Fix 600 permissions on /etc/central-backup/*
    
Options:
  -h --help     Show this screen.
  -n --hosts HOSTS     Host(,) to backup. Comma for multiple hosts
  -d --databases DB   Database to backup. Comma for multiple DBs
  -f --features FEATURES    Features to execute on backup (eg: databases, files)

Backup repository Config

BACKUP_DIRECTORY: /my/backup/directory/
NOTIFICATIONS:
  from: user@domain.com
  to: user@domain.com
  level: error  # ERROR or INFO
  server: mail.domain.com  # SSL support mandatory
  pogin: user
  password: password
RETENTION:
  months: 120
  weeks: 4
  days: 7
SCHEDULE:
  default_hour: 01:00
  default_day: monday

Host config

Per host configuration take places in :

/etc/central-backups/conf.d

--- my_host.yml ---
hostname: my_host_name 
# hostname with dns resolution (local or remote)
working_dir: /home/user/backup 
# place on host for dump data (automatic creation)
user: user
# ssh user for connection. Make sudo ssh-copy-id for it.
sudo_password: my_sudo (optional)
# if need for some file access.
schedule: daily 
# scheduling, see below for possible options.
databases: 
#databases section to list db to backup
  - db: my_db_name
    dbtype: my_db_type (psql|mysql)
    user: my_db_user
    password: my_db_password
    exclude: [] # table to exclude, not working now
files: 
#files section to list directories to backup
  - /etc/supervisor
  - /etc/nginx/sites-enabled
  - /etc/nginx/sites-available
  - /etc/nginx/cert

Host config : schedule format

every|at|option:option_value::other_option:other_option_value

# special format
schedule: (daily|weekly|monthly|hourly) 

# cron format
schedule: "15 10 * * *"

# multiple schedule, custom format
# every|at|option:option_value::other_option:other_option_value
schedule:  
  - 6.hour|minute.15|databases:mydatabases::features:databases
  - 1.day|minute.15|features:files

Workflow

TODO

• override retention per host
• restore features
• tables to exclude
• make tests

Credits

• fabric, patchwork : Simple, Pythonic remote execution and deployment.
• pyyaml : YAML parser and emitter for Python
• coloredlogs : Colored terminal output for Python’s logging module.
• dateparser : Python parser for human readable dates.
• jinja2: Jinja is a modern and designer-friendly templating language for Python, modelled after Django’s templates.
• simplesmtp: A very simple Python class to send e-mails using the SMTP Python Lib
• python-slugify : Returns unicode slugs
• docopt : Arguments that make me smile.
• plan : Crontab jobs management in Python