You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Thierry BOULOGNE 966c9249ac debug 1 year ago
backup debug 1 year ago
scripts add checmount 1 year ago
.gitignore add swift 1 year ago
LICENSE first commit 2 years ago
MANIFEST.in add checmount 1 year ago
README.md gg 2 years ago
central-backup.yaml add swift 1 year ago
check_mount add checmount 1 year ago
host.yml first commit 2 years ago
script_template.py report space 1 year ago
setup.py debug 1 year ago

README.md

Features

  • per host backup
  • per group backup
  • per features (database | files) backups
  • schedule backup with automatic cron generation
  • simple configuration via yaml file
  • command line utility
  • retention period
  • backup reports with success / failed

Setup

  1. make and activate a virtualenv
  2. pip install git+https://git.tng-concepts.com/TNG/central-backup

Sudo password will be ask to

  • copy central_backup utility in /usr/local/bin.
  • create /etc/central-backup directories
  • create a default config files
  1. create your hosts to backup under /etc/central-backup/conf.d (see below)
  2. fill /etc/central-backup/config.yml
  3. Run central_backup fix (fix permissions) and central_backup schedule (crontab creation)

You got it !

Command line usage

Python utility to manage centralized backups with ssh.

Usage: 
  central_backup (backup | restore | fix | schedule) [options]
  central_backup (backup | restore) (--databases=<STR> --hosts=<STR>) [--features=<STR>]
  central_backup (backup | restore) [--hosts=<STR>] [--features=<STR>]

Commands:
    backup  Launch backup action.
    restore Launch restore action
    schedule Crontab creation regarding schedule information in host files.
    fix     Fix 600 permissions on /etc/central-backup/*
    
Options:
  -h --help     Show this screen.
  -n --hosts HOSTS     Host(,) to backup. Comma for multiple hosts
  -d --databases DB   Database to backup. Comma for multiple DBs
  -f --features FEATURES    Features to execute on backup (eg: databases, files)

Backup repository Config

BACKUP_DIRECTORY: /my/backup/directory/
NOTIFICATIONS:
  from: user@domain.com
  to: user@domain.com
  level: error  # ERROR or INFO
  server: mail.domain.com  # SSL support mandatory
  pogin: user
  password: password
RETENTION:
  months: 120
  weeks: 4
  days: 7
SCHEDULE:
  default_hour: 01:00
  default_day: monday

Host config

Per host configuration take places in :

/etc/central-backups/conf.d

--- my_host.yml ---
hostname: my_host_name 
# hostname with dns resolution (local or remote)
working_dir: /home/user/backup 
# place on host for dump data (automatic creation)
user: user
# ssh user for connection. Make sudo ssh-copy-id for it.
sudo_password: my_sudo (optional)
# if need for some file access.
schedule: daily 
# scheduling, see below for possible options.
databases: 
#databases section to list db to backup
  - db: my_db_name
    dbtype: my_db_type (psql|mysql)
    user: my_db_user
    password: my_db_password
    exclude: [] # table to exclude, not working now
files: 
#files section to list directories to backup
  - /etc/supervisor
  - /etc/nginx/sites-enabled
  - /etc/nginx/sites-available
  - /etc/nginx/cert

Host config : schedule format

every|at|option:option_value::other_option:other_option_value

# special format
schedule: (daily|weekly|monthly|hourly) 

# cron format
schedule: "15 10 * * *"

# multiple schedule, custom format
# every|at|option:option_value::other_option:other_option_value
schedule:  
  - 6.hour|minute.15|databases:mydatabases::features:databases
  - 1.day|minute.15|features:files

Workflow

TODO

  • override retention per host
  • restore features
  • tables to exclude
  • make tests

Credits

  • fabric, patchwork : Simple, Pythonic remote execution and deployment.
  • pyyaml : YAML parser and emitter for Python
  • coloredlogs : Colored terminal output for Python’s logging module.
  • dateparser : Python parser for human readable dates.
  • jinja2: Jinja is a modern and designer-friendly templating language for Python, modelled after Django’s templates.
  • simplesmtp: A very simple Python class to send e-mails using the SMTP Python Lib
  • python-slugify : Returns unicode slugs
  • docopt : Arguments that make me smile.
  • plan : Crontab jobs management in Python