Git Product home page Git Product logo

rsnapshot-docker-compose-backup's Introduction

rsnapshot-docker-compose-backup

This program can be used to create config to back up docker compose container with rsnapshot.

Get Started

Prerequisites

This script expects a folder structure in the following scheme:

docker-compose-root/  
  - projectDir1/  
      - docker-compose.yml  
  - projectDir2/  
      - docker-compose.yml  
      - ...  
  - ...

The config file for this script has to be in the docker compose root.

Quick Start

  1. Execute the Script
  2. Examine Output
  3. Change newly generated backup.ini in docker compose root directory if necessary
  4. Create backup.ini files in projectDirs that need special configuration

Steps

The Backup process is divided in steps. There are 7 Steps:

  • runtime_backup
  • pre_stop
  • stop
  • pre_backup
  • backup
  • post_backup
  • restart
  • post_restart

The steps are executed in that order.
The Idea is that the first step runtime_backup can be used for everything where the service can still be running, like backup of the image.
The stop step can be used to stop the container, so that the backup step can be used for a backup of the volumes.
After that the container can be restarted in the restart step.

Every command that gets executed in one of the steps is defined in the config file and can be changed fit your need.

Config File

This script is completely controlled via config files. There are two different type of config files:

  1. global config
  2. project level config

The global config is created with the first start of the script. It contains the default configuration for all projects and some more settings.
The project level config is not necessary but can be used to override the default configuration for individual services.

All config files are named backup.ini. The config file is divided in different sections and all entries are in the key=value schema like in other .ini files. All keys and sections are case-insensitive.

Default files

Global config

#Start of main section
[default_config]
    #Specific commands for each step
    backup = backup	$projectFolder	$backupPrefixFolder/$serviceName/projectDir
    
    [default_config.actions]
        stopContainer = true
        volumeBackup = true
        yamlBackup = false
    [default_config.vars]
        prefix = docker-compose

[predefined_actions]
    [predefined_actions.volumeBackup]
        backup = backup	$volumes.path	$prefix/$serviceName/$volumes.name
    
    [predefined_actions.yamlBackup]
        runtime_backup = backup	$projectFolder	$prefix/$serviceName/yaml	+rsync_long_args=--include=*.yml,+rsync_long_args=--include=*.yaml
    
    [predefined_actions.stopContainer]
        stop = backup_exec	cd $projectFolder; /usr/bin/docker-compose stop
        restart = backup_exec	cd $projectFolder; /usr/bin/docker-compose start

Project level config

[Nextcloud]
    pre_backup=backup_exec  docker exec -u www-data nextcloud /var/www/html/occ maintenance:mode --on
    post_backup=backup_exec docker exec -u www-data nextcloud /var/www/html/occ maintenance:mode --off
    
    [Nextcloud.Actions]
        yamlBackup=true
        stopContainer=false
[MariaDB]
    runtime_backup=backup_script   $projectFolder/dump_database.sh    /backupDir
    pre_stop=backup_exec  docker exec -u www-data nextcloud /var/www/html/occ maintenance:mode --on
    post_restart=backup_exec docker exec -u www-data nextcloud /var/www/html/occ maintenance:mode --off

Sections

The global config file has to main sections: - default_config - predefined_actions

The project level config file has a main section for every service that should be specially configured.

The main sections from the project level config and the default_config section of the global_config have the same structure.

It is possible to define commands that should be executed directly in this section. The commands are defined with the step, where they apply, as key and the rsnapshot command as value. The command can be multiple lines long.

The section also has different subsections. The most used subsection is called {main_section}.Actions. This Section defines which of the actions that are defined in the global config should be activated or deactivated. That can be done with the action Name as Key and true or false as value.

The second subsection has the name {main_section}.Vars and is used to define variables that can be used in rsnapshot commands. Variables can be defined with {var_name}={value}. The Variables can only be used for simple text replacement and no complex actions. Some variables are predefined. They are listed in the default Variables section.

In the global config is a second main section with the name predefined_actions. This section contains actions that can be activated in the before explained .Actions section. The actions can be used to modularise the commands and for a possibility to disable actions on service level that are activated in the global config.

Every action has an own section with the name prefefined_actions.{action_name}. This section can contain RSnapshot commands like the other main section. It is also possible that an action has subsections that are defined like this: actions={one ore more actions}. These subsections can only be one level deep.

Default Variables

Some Variables are predefined and can be used in the backup.ini without being defined explicitly. These are the variables:

  • $serviceName: The name of the Service
  • $containerName: The name of the Container
  • $projectName: The name of the Project
  • $containerID: The ID of the Container
  • $projectFolder: The Path of the Project Folder
  • $volumes: A List of the volumes that are defined for the service. The backup commands are copied for each volume.

rsnapshot-docker-compose-backup's People

Contributors

asdfgamer avatar ku-we avatar

Stargazers

 avatar

Watchers

 avatar  avatar

rsnapshot-docker-compose-backup's Issues

Use Docker Python Api

To improve the performance and redability of the Backup the python Libary from Docker should be used.
This libary is automaticly installed if docker-compose is installed, but in a few cases this libary is not in the import path, for example in virtual environments. Another reason could be, that a old version of docker-compose is installed that uses the Python2 Version of the libary.
To solve this Problem the current path should exist as a fallback alternative to find the container.

Convert backup_script to backup_exec and backup

A setting to convert backup_script calls to backup_exec and backup calls would be very helful in mutliple situations:

  1. backup calls have a better log output
  2. backup calls support incremental backups

The problem with this setting is, that the files that are created/changed by backup_script have to be recognized and copied to a temp folder. This is probably relativly complicated.

Release Package on PyPI

The Package should be released on PyPI. The Package should automatically be updated when a new release is created.

FR: Support all (four) valid names for compose files

The Compose Specification defines four valid names for compose files (https://docs.docker.com/compose/compose-application-model/).

The default path for a Compose file is compose.yaml (preferred) or compose.yml that is placed in the working directory. Compose also supports docker-compose.yaml and docker-compose.yml for backwards compatibility of earlier versions. If both files exist, Compose prefers the canonical compose.yaml.

As of now, only docker-compose.yml seems to be supported.

if "docker-compose.yml" in tree_element[2]:

All names matching the regular expression (docker-)?compose\.y(a)?ml should be suuported.

FR: Support $containerName as variable

We already support $serviceName, $containerID, $projectFolder, $volumes, $image and $projectName as variables in backup.ini.

The new variable $containerName would be useful in backup destination path names because it is unique (in contrast to the default $serviceName). The docker daemon does not allow two containers with the same name running at the same time.

The container's name is auto-generated by docker but can be overridden by setting container_name in the compose file.

Set Project Name

Currently this script finds only docker-compose container where the compose-project-name equals the folder name. This is the default.
It would be good to add an setting to change the project name

Update Readme

The Readme schould contain install instructions and the old parts of the readme should be deleted

1.0 release

The following things have to be done before the 1.0 release:

  • Release Package on PyPI (see #18 )
  • Test in a different Environment (@ku-we)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.