Git Product home page Git Product logo

backup-service's Introduction

Photography Backup

Docker container that uses Borg and rclone to create backups deployed in Google Cloud Storage.

Requirements

  • A Google account with a GCP project (billing must be enabled)
  • gcloud CLI to configure your GCP authentication
  • terraform to provision the required resources on GCP
  • docker to build and run the backup container

Configure Cloud Storage

Before using this project, you have to configure your GCP authentication using gcloud CLI:

gcloud auth application-default login

Create a .auto.tfvars file inside provision/ directory with the following variables set:

google_project_id = "<GCP_PROJECT_ID>"
google_region     = "<GCP_REGION>"
google_zone       = "<GCP_ZONE>"

Initialize Terraform locally and start the provisioning:

cd provision
terraform init
terraform apply

Once the provision ends, you can re-initialize Terraform to move the global state in your newly created buckets. To do that, configure your backend by creating a .gcs.tfbackend file inside provision/ directory with the following variables set:

bucket = "<YOUR_BUCKET_NAME>"
prefix = "terraform/state"

Then, re-initialize Terraform to move the state on Google Cloud Storage:

terraform init -backend-config=.gcs.tfbackend

Backup Service

Configure Backup Environment

To boostrap your environment, launch the following commands from the repository folder:

mkdir -p config/
mkdir -p cache/
cp env.example .env
chmod 600 .env

Docker Compose uses automatically the .env file. You must update included values as follows:

  • DATA_PATH: is the folder you want to backup. Example: /home/user/Pictures.
  • BORG_PASSPHRASE: is the key used to encrypt Borg repository. Check Borg documentation for more details.
  • RCLONE_REMOTE_PATH: is the remote storage path in rcloud format. Example: REMOTE_NAME:BUCKET_NAME.
  • CROND_SCHEDULE: is the schedule used by crond to execute the backup. Example: 0 2 * * * (every day at 2 AM).

Start the Container

Once the configuration is done, you can run the container as a service that will execute the backup based on the CROND_SCHEDULE. If the backup is not initialized, the container will create a new repository in the rclone mount folder.

To run the container as a service, use the following command:

docker compose up -d

Extract the Archive

To extract the backup archive you must stop any running container. Once the container is stopped, you can use the following helper:

docker compose run --rm -ti backup extract.sh

Development

We accept external contributions even though the project is mostly designed for personal needs. If you think some parts can be exposed with a more generic interface, feel free to open a GitHub issue to discuss your suggestion.

backup-service's People

Contributors

palazzem avatar

Stargazers

Fabrizio Buratta avatar

Watchers

James Cloos avatar  avatar  avatar

backup-service's Issues

Make the project generic

Overview

This project is tailored to my specific use case and at the time of writing I wasn't planning to make it generic. You can use this placeholder to kick-off the conversation if you are interested on it.

Handle SIGTERM signals

Overview

If the container is interrupted (e.g. docker stop), borg lock files are not closed, causing issues during the next run. When the container stops it should clean-up eventual lock files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.