[Feature Request] Support for backing up docker bind mounts about autorestic HOT 11 CLOSED

Maybe it could work in the form of a command to launch before/after the backup?

locations:
  docker-postgres:
    from: /srv/docker-postgres/data
    to: storage
    run_before: docker stop postgres
    run_after: docker start postgres

this way it isn't specific to docker, and works for other use cases?

from autorestic.

It makes the tool way more flexible. I'll implement it as soon as I have time :)

from autorestic.

@FarisZR you can do that exactly with the before and after hooks:

locations:
  mydb-backup:
    from:  /home/user/postgres
    hooks:
      before:
        - sudo docker stop my-db
      after:
        - sudo docker start my-db

EDIT: a better option to avoid downtime might be not to stop the container at all and use pg_dumpall, e.g :

locations:
  mydb-backup:
    from:  /var/backups/postgres
    hooks:
      before:
        - mkdir -p /var/backups/postgres
        - sudo docker run my-db pg_dumpall > /var/backups/postgres/backup.sql
      after:
        - rm /var/backups/postgres/backup.sql

from autorestic.

This WE I'll be thinking about some potential ways to backup docker volumes. I assume you mean volumes that are not mapped to the local file system :)

Good suggestion!

from autorestic.

@EliotBerriot your proposed hook looks great for CLI support; docker, kubectl, service, systemctl, etc. would have basic support with this.

Just as an FYI, referencing similar functionality in other provisioning apps can be helpful, e.g. https://github.com/ansible/ansible-modules-core/blob/devel/system/service.py to avoid common pitfalls with sleep/timeouts/status codes/etc.

@cupcakearmy I primarily use local file system mounts (or NFS, etc.) and my concern is stopping apps to keep data integrity. Of course first-class support for docker volumes (and/or other management APIs) would be fantastic. A basic shell hook would go a long way to supporting more cases.

from autorestic.

https://github.com/cupcakearmy/autorestic#before--after-hooks

I still need to think about what the best way is to integrate docker volumes. Any suggestions are welcome :)

from autorestic.

# Backup
docker run --rm --volume my_volume_to_backup:/data -v $(pwd):/backup alpine tar cf /backup/backup.tar -C /data .

# Restore
docker run --rm --volume my_volume_to_backup:/data -v $(pwd):/backup alpine tar xf /backup/backup.tar -C /data

This could be the way to go

from autorestic.

So, I tried a few things

1

Mount the volume -> copy data to host -> run restic

This worked but it had the problem that docker will modify permissions and groups of the files -> no,no
(Tried with cp -aT to preserve permissions)

2

Then i tried to do the same thing bypassing the mounted volume permissions limitation with tar

Mount volume -> tar -> copy tar to host -> untar in host -> run restic

This unfortunately did work but again the permissions were lost. My guess is that if tar does not find the user when decompressing it will replace it. Not sure..

3

Mount volume -> tar -> copy tar to host -> run restic

This approach works perfectly but it does not allow incremental updates as restic just sees the tarbal. It's not the best thing, but at least it guarantees that permissions are preserved 100%

So I implemented version 3. Will be released soon :)

from autorestic.

It's done 🚀 docker volumes can now be backed up and restored with out needing them to be mounted to the host.

@jacobblock @EliotBerriot

https://github.com/cupcakearmy/autorestic/blob/master/README.md#-docker-volumes

from autorestic.

can i backup mounted volumes using autorestic in docker ?
for example, i have autorestic running in docker.
and the container is postgres running in /home/user/postgres.
i mount /home/user/postgres to autorestic, and then it stops the container temporarily while the backup is running.

is this possible ? or does it need autorestic running outside of docker to execute commands to stop/start the container ?

thanks for the awesome project @cupcakearmy !

from autorestic.

from autorestic.