You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Thierry BOULOGNE 966c9249ac debug 8 months ago
backup debug 8 months ago
scripts add checmount 11 months ago
.gitignore add swift 11 months ago
LICENSE first commit 11 months ago
MANIFEST.in add checmount 11 months ago
README.md gg 11 months ago
central-backup.yaml add swift 11 months ago
check_mount add checmount 11 months ago
host.yml first commit 11 months ago
script_template.py report space 11 months ago
setup.py debug 8 months ago

README.md

Features

  • per host backup
  • per group backup
  • per features (database | files) backups
  • schedule backup with automatic cron generation
  • simple configuration via yaml file
  • command line utility
  • retention period
  • backup reports with success / failed

Setup

  1. make and activate a virtualenv
  2. pip install git+https://git.tng-concepts.com/TNG/central-backup

Sudo password will be ask to

  • copy central_backup utility in /usr/local/bin.
  • create /etc/central-backup directories
  • create a default config files
  1. create your hosts to backup under /etc/central-backup/conf.d (see below)
  2. fill /etc/central-backup/config.yml
  3. Run central_backup fix (fix permissions) and central_backup schedule (crontab creation)

You got it !

Command line usage

Python utility to manage centralized backups with ssh.

Usage: 
  central_backup (backup | restore | fix | schedule) [options]
  central_backup (backup | restore) (--databases=<STR> --hosts=<STR>) [--features=<STR>]
  central_backup (backup | restore) [--hosts=<STR>] [--features=<STR>]

Commands:
    backup  Launch backup action.
    restore Launch restore action
    schedule Crontab creation regarding schedule information in host files.
    fix     Fix 600 permissions on /etc/central-backup/*
    
Options:
  -h --help     Show this screen.
  -n --hosts HOSTS     Host(,) to backup. Comma for multiple hosts
  -d --databases DB   Database to backup. Comma for multiple DBs
  -f --features FEATURES    Features to execute on backup (eg: databases, files)

Backup repository Config

BACKUP_DIRECTORY: /my/backup/directory/
NOTIFICATIONS:
  from: user@domain.com
  to: user@domain.com
  level: error  # ERROR or INFO
  server: mail.domain.com  # SSL support mandatory
  pogin: user
  password: password
RETENTION:
  months: 120
  weeks: 4
  days: 7
SCHEDULE:
  default_hour: 01:00
  default_day: monday

Host config

Per host configuration take places in :

/etc/central-backups/conf.d

--- my_host.yml ---
hostname: my_host_name 
# hostname with dns resolution (local or remote)
working_dir: /home/user/backup 
# place on host for dump data (automatic creation)
user: user
# ssh user for connection. Make sudo ssh-copy-id for it.
sudo_password: my_sudo (optional)
# if need for some file access.
schedule: daily 
# scheduling, see below for possible options.
databases: 
#databases section to list db to backup
  - db: my_db_name
    dbtype: my_db_type (psql|mysql)
    user: my_db_user
    password: my_db_password
    exclude: [] # table to exclude, not working now
files: 
#files section to list directories to backup
  - /etc/supervisor
  - /etc/nginx/sites-enabled
  - /etc/nginx/sites-available
  - /etc/nginx/cert

Host config : schedule format

every|at|option:option_value::other_option:other_option_value

# special format
schedule: (daily|weekly|monthly|hourly) 

# cron format
schedule: "15 10 * * *"

# multiple schedule, custom format
# every|at|option:option_value::other_option:other_option_value
schedule:  
  - 6.hour|minute.15|databases:mydatabases::features:databases
  - 1.day|minute.15|features:files

Workflow

TODO

  • override retention per host
  • restore features
  • tables to exclude
  • make tests

Credits

  • fabric, patchwork : Simple, Pythonic remote execution and deployment.
  • pyyaml : YAML parser and emitter for Python
  • coloredlogs : Colored terminal output for Python’s logging module.
  • dateparser : Python parser for human readable dates.
  • jinja2: Jinja is a modern and designer-friendly templating language for Python, modelled after Django’s templates.
  • simplesmtp: A very simple Python class to send e-mails using the SMTP Python Lib
  • python-slugify : Returns unicode slugs
  • docopt : Arguments that make me smile.
  • plan : Crontab jobs management in Python