loomchild / volume-backup Goto Github PK
View Code? Open in Web Editor NEWdocker volume backup & restore utility
License: MIT License
docker volume backup & restore utility
License: MIT License
I am using Windows Docker Desktop (WSL2 Engine, PowerShell to run commands) and I used volume-backup previously to create a backup of my volume using the following command:
.
docker run --rm -v 8c50ada51b75c5906b5dd008a1a1065c06c24c7b4f88405deaa7bb73907d64b7:/volume -v c:/haproxy:/backup loomchild/volume-backup backup nextcloud_backup
.
The file that was created had a name: nextcloud_backup.tar.bz2
.
After server change I cannot restore my volume. I am trying to usethe following commands:
docker run --rm -v c:/Docker_Data:/volume -v /tmp:/backup loomchild/volume-backup restore nextcloud_backup.tar.bz2
docker run --rm -v c:/Docker_Data:/volume -v /tmp:/backup loomchild/volume-backup restore C:\Docker_Data\nextcloud_backup.tar.bz2
docker run --rm -v c:/Docker_Data:/volume -v /tmp:/backup loomchild/volume-backup restore C:/Docker_Data/nextcloud_backup.tar.bz2
docker run --rm -v 8c50ada51b75c5906b5dd008a1a1065c06c24c7b4f88405deaa7bb73907d64b7:/volume -v /tmp:/backup loomchild/volume-backup restore C:/Docker_Data/nextcloud_backup.tar.bz2
Every time I receive the following response:
Archive file C:/Docker_Data/nextcloud_backup.tar.bz2 does not exist
If we do docker volume ls -q, we're able to see all volumes currently on Docker.
If we can do something like
docker container stop $(docker container ls -aq)
backup command with docker volume -q as a variable
docker container start $(docker container ls -aq)
We should be able to stop all containers, backup their respective volumes, and restart them.
Thoughts?
Hello,
thank you for your work, I'm using it a lot 👍
Is there a way to exclude a directory from the backup of a volume.
Thanks
The stdout backup appears to generate an uncompressed output, but the stdin input requires the input as bz2. This seems inconsistent and confusing.
Would it be possible to support pbzip2 (plus maybe nice and ionice)?
Should have an option to quit if there is an existing volume of the same name.
why not use this options to mount volume from source docker ?
Issue docker/compose#4738 has been closed as no longer reproducible, so I presume this means the one-second sleep is no longer required.
I'm currently trying to write a backup wrapper around this docker image (which has been very helpful, thanks!) and have started to run into a roadblock - when it gets to backing up my gitlab_data volume (which weighs in at 4.025GB, per docker system df -v
), I run into this:
Jan 14 14:26:13 nomad volume-backup.sh[9945]: bzip2: short write
Jan 14 14:26:13 nomad volume-backup.sh[9945]: tar: write error: Broken pipe
from which it turns into an unrecoverable failure and the backup cannot be completed. I'm not sure if it's just too many file handles or what; the only thing of relevance that I have found so far is this:
https://askubuntu.com/questions/835226/error-with-gzip-while-used-along-with-tar
When you backup a volume (for example /etc/foo) and this folder has the chmod permissions 700, the folder has different permissions after restoration.
This becomes evident, when backing up a postgres volume, as postgresl won't start when the folder has wrong permissions.
Just tried with Docker on Windows 10 Pro. Got a perfect .tar.bz2
archive.
docker run -v my-volume:/volume --rm loomchild/volume-backup backup - > some_archive.tar.bz2
Thanks!
By looking at the forks, I noticed that many of them only modify compression algorithm, and these two are the most popular and easiest to configure.
E.g.
papeng@1ca8030
https://github.com/githubsaturn/volume-backup/commit/116e83585f2c4895c675e170feffc597e44547f2
Is there any way to add a “verbose” option to the backup/restore, or a status indicator? A few times I’ve been backing up a big volume and with no output, I didn’t know if the process was stalled, crashed or just slow and not finished yet… it doesn't even need to be a percentage indicator, just some sort of indication that the process is still running and hasn't crashed, a stdout of a '.' every 10/20/30secs even would do....
I had a try to take the docker persistent volume back up with the utility loomchild/volume-backup
but seems it did not work. I had 800GB data but both backup & restore was pretty quick (~4 mins). When I deleted the actual data from the hard disk I understood the utility didn't work as expected.
It's working fine with the small volumes. Is this due to the default block size limit which's 10GB? Does docker persistent volume automatically create the new volume in case the volume size exceeds ~10GB?
I noticed that if I have a typo in the volume name, volume-backup will complete but the backup will be empty. Could backup (and restore) warn (or even die) if the volume is empty (newly created)?
I have copied the bz2 file to another machine and tried to restore it. But I get an error there is not sufficient space error. Any clues here.
Hey, I think this Dockerfile might not work on Raspberry Pi 4. The architecture is aarch64. When I run:
docker run -v verdaccio_verdaccio:/volume -v /tmp:/backup --rm loomchild/volume-backup backup test
OR
docker run -v verdaccio_verdaccio:/volume -v /tmp:/backup --rm loomchild/volume-backup backup -> verdaccio_verdaccio.tar.bz2
I get the following error:
standard_init_linux.go:211: exec user process caused "exec format error"
Any idea on how to get this working? If not, any idea on what alternative I should be using?
Hi there,
Love this tool! I was using it without issue until yesterday; I now receive this error upon running which is difficult to debug:
[centos@oc]$ docker run -v owncloud_mysql:/volume --rm loomchild/volume-backup backup -| aws s3 cp - s3://{REMOVED-FOR-GITHUB}/owncloud_mysql-$(date +%F_%H-%M-%S).tar.bz2 --storage-class STANDARD_IA
Traceback (most recent call last):
File "/usr/bin/aws", line 19, in <module>
import awscli.clidriver
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 17, in <module>
import botocore.session
File "/usr/lib/python2.7/site-packages/botocore/session.py", line 29, in <module>
import botocore.configloader
File "/usr/lib/python2.7/site-packages/botocore/configloader.py", line 19, in <module>
from botocore.compat import six
File "/usr/lib/python2.7/site-packages/botocore/compat.py", line 26, in <module>
from urllib3 import exceptions
File "/usr/lib/python2.7/site-packages/urllib3/__init__.py", line 10, in <module>
from .connectionpool import (
File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 31, in <module>
from .connection import (
File "/usr/lib/python2.7/site-packages/urllib3/connection.py", line 45, in <module>
from .util.ssl_ import (
File "/usr/lib/python2.7/site-packages/urllib3/util/__init__.py", line 4, in <module>
from .request import make_headers
File "/usr/lib/python2.7/site-packages/urllib3/util/request.py", line 5, in <module>
from ..exceptions import UnrewindableBodyError
ImportError: cannot import name UnrewindableBodyError
write /dev/stdout: broken pipe
As you can see, I'm backing up a mysql volume on a centos box. The only things that have changed (though I'm not sure they'd impact the run) are:
Packages Altered:
Updated GeoIP-1.5.0-13.el7.x86_64 ?
Update 1.5.0-14.el7.x86_64 @base
Updated audit-2.8.4-4.el7.x86_64 ?
Update 2.8.5-4.el7.x86_64 @base
Updated audit-libs-2.8.4-4.el7.x86_64 ?
Update 2.8.5-4.el7.x86_64 @base
Updated audit-libs-python-2.8.4-4.el7.x86_64 ?
Update 2.8.5-4.el7.x86_64 @base
Updated bash-4.2.46-31.el7.x86_64 ?
Update 4.2.46-33.el7.x86_64 @base
Dep-Install bind-export-libs-32:9.11.4-9.P2.el7.x86_64 @base
Updated bind-libs-32:9.9.4-74.el7_6.2.x86_64 @updates
Update 32:9.11.4-9.P2.el7.x86_64 @base
Updated bind-libs-lite-32:9.9.4-74.el7_6.2.x86_64 ?
Update 32:9.11.4-9.P2.el7.x86_64 @base
Updated bind-license-32:9.9.4-74.el7_6.2.noarch ?
Update 32:9.11.4-9.P2.el7.noarch @base
Updated bind-utils-32:9.9.4-74.el7_6.2.x86_64 @updates
Update 32:9.11.4-9.P2.el7.x86_64 @base
Updated binutils-2.27-34.base.el7.x86_64 ?
Update 2.27-41.base.el7.x86_64 @base
Updated centos-release-7-6.1810.2.el7.centos.x86_64 ?
Update 7-7.1908.0.el7.centos.x86_64 @base
Updated chrony-3.2-2.el7.x86_64 ?
Update 3.4-1.el7.x86_64 @base
Updated cloud-init-18.2-1.el7.centos.2.x86_64 ?
Update 18.5-3.el7.centos.x86_64 @base
Updated cloud-utils-growpart-0.29-2.el7_6.2.noarch ?
Update 0.29-5.el7.noarch @base
Updated container-selinux-2:2.107-1.el7_6.noarch @extras
Update 2:2.107-3.el7.noarch @extras
Updated coreutils-8.22-23.el7.x86_64 ?
Update 8.22-24.el7.x86_64 @base
Updated cronie-1.4.11-20.el7_6.x86_64 ?
Update 1.4.11-23.el7.x86_64 @base
Updated cronie-anacron-1.4.11-20.el7_6.x86_64 ?
Update 1.4.11-23.el7.x86_64 @base
Updated cryptsetup-libs-2.0.3-3.el7.x86_64 ?
Update 2.0.3-5.el7.x86_64 @base
Updated curl-7.29.0-51.el7_6.3.x86_64 ?
Update 7.29.0-54.el7.x86_64 @base
Updated device-mapper-7:1.02.149-10.el7_6.8.x86_64 ?
Update 7:1.02.158-2.el7.x86_64 @base
Updated device-mapper-libs-7:1.02.149-10.el7_6.8.x86_64 ?
Update 7:1.02.158-2.el7.x86_64 @base
Updated dhclient-12:4.2.5-68.el7.centos.1.x86_64 ?
Update 12:4.2.5-77.el7.centos.x86_64 @base
Updated dhcp-common-12:4.2.5-68.el7.centos.1.x86_64 ?
Update 12:4.2.5-77.el7.centos.x86_64 @base
Updated dhcp-libs-12:4.2.5-68.el7.centos.1.x86_64 ?
Update 12:4.2.5-77.el7.centos.x86_64 @base
Updated diffutils-3.3-4.el7.x86_64 ?
Update 3.3-5.el7.x86_64 @base
Updated dmidecode-1:3.1-2.el7.x86_64 ?
Update 1:3.2-3.el7.x86_64 @base
Updated dracut-033-554.el7.x86_64 ?
Update 033-564.el7.x86_64 @base
Updated dracut-config-generic-033-554.el7.x86_64 ?
Update 033-564.el7.x86_64 @base
Updated dracut-config-rescue-033-554.el7.x86_64 ?
Update 033-564.el7.x86_64 @base
Updated dracut-network-033-554.el7.x86_64 ?
Update 033-564.el7.x86_64 @base
Updated e2fsprogs-1.42.9-13.el7.x86_64 ?
Update 1.42.9-16.el7.x86_64 @base
Updated e2fsprogs-libs-1.42.9-13.el7.x86_64 ?
Update 1.42.9-16.el7.x86_64 @base
Updated elfutils-default-yama-scope-0.172-2.el7.noarch ?
Update 0.176-2.el7.noarch @base
Updated elfutils-libelf-0.172-2.el7.x86_64 ?
Update 0.176-2.el7.x86_64 @base
Updated elfutils-libs-0.172-2.el7.x86_64 ?
Update 0.176-2.el7.x86_64 @base
Updated ethtool-2:4.8-9.el7.x86_64 ?
Update 2:4.8-10.el7.x86_64 @base
Updated freetype-2.8-12.el7_6.1.x86_64 ?
Update 2.8-14.el7.x86_64 @base
Dep-Install geoipupdate-2.5.0-1.el7.x86_64 @base
Updated glib2-2.56.1-4.el7_6.x86_64 ?
Update 2.56.1-5.el7.x86_64 @base
Updated glibc-2.17-260.el7_6.6.x86_64 ?
Update 2.17-292.el7.x86_64 @base
Updated glibc-common-2.17-260.el7_6.6.x86_64 ?
Update 2.17-292.el7.x86_64 @base
Updated grub2-1:2.02-0.76.el7.centos.1.x86_64 ?
Update 1:2.02-0.80.el7.centos.x86_64 @base
Updated grub2-common-1:2.02-0.76.el7.centos.1.noarch ?
Update 1:2.02-0.80.el7.centos.noarch @base
Updated grub2-pc-1:2.02-0.76.el7.centos.1.x86_64 ?
Update 1:2.02-0.80.el7.centos.x86_64 @base
Updated grub2-pc-modules-1:2.02-0.76.el7.centos.1.noarch ?
Update 1:2.02-0.80.el7.centos.noarch @base
Updated grub2-tools-1:2.02-0.76.el7.centos.1.x86_64 ?
Update 1:2.02-0.80.el7.centos.x86_64 @base
Updated grub2-tools-extra-1:2.02-0.76.el7.centos.1.x86_64 ?
Update 1:2.02-0.80.el7.centos.x86_64 @base
Updated grub2-tools-minimal-1:2.02-0.76.el7.centos.1.x86_64 ?
Update 1:2.02-0.80.el7.centos.x86_64 @base
Updated grubby-8.28-25.el7.x86_64 ?
Update 8.28-26.el7.x86_64 @base
Updated gssproxy-0.7.0-21.el7.x86_64 ?
Update 0.7.0-26.el7.x86_64 @base
Updated hwdata-0.252-9.1.el7.x86_64 ?
Update 0.252-9.3.el7.x86_64 @base
Updated initscripts-9.49.46-1.el7.x86_64 ?
Update 9.49.47-1.el7.x86_64 @base
Updated iproute-4.11.0-14.el7_6.2.x86_64 ?
Update 4.11.0-25.el7.x86_64 @base
Updated iptables-1.4.21-28.el7.x86_64 ?
Update 1.4.21-33.el7.x86_64 @base
Updated irqbalance-3:1.0.7-11.el7.x86_64 ?
Update 3:1.0.7-12.el7.x86_64 @base
Updated iwl7265-firmware-22.0.7.0-69.el7.noarch ?
Update 22.0.7.0-72.el7.noarch @base
Install kernel-3.10.0-1062.1.1.el7.x86_64 @updates
Updated kernel-tools-3.10.0-957.27.2.el7.x86_64 ?
Update 3.10.0-1062.1.1.el7.x86_64 @updates
Updated kernel-tools-libs-3.10.0-957.27.2.el7.x86_64 ?
Update 3.10.0-1062.1.1.el7.x86_64 @updates
Updated kexec-tools-2.0.15-21.el7_6.4.x86_64 ?
Update 2.0.15-33.el7.x86_64 @base
Updated kmod-20-23.el7.x86_64 ?
Update 20-25.el7.x86_64 @base
Updated kmod-libs-20-23.el7.x86_64 ?
Update 20-25.el7.x86_64 @base
Updated kpartx-0.4.9-123.el7.x86_64 ?
Update 0.4.9-127.el7.x86_64 @base
Updated krb5-libs-1.15.1-37.el7_6.x86_64 @?base
Update 1.15.1-37.el7_7.2.x86_64 @updates
Updated libblkid-2.23.2-59.el7_6.1.x86_64 ?
Update 2.23.2-61.el7.x86_64 @base
Updated libcap-2.22-9.el7.x86_64 ?
Update 2.22-10.el7.x86_64 @base
Updated libcgroup-0.41-20.el7.x86_64 ?
Update 0.41-21.el7.x86_64 @base
Updated libcom_err-1.42.9-13.el7.x86_64 ?
Update 1.42.9-16.el7.x86_64 @base
Updated libcurl-7.29.0-51.el7_6.3.x86_64 ?
Update 7.29.0-54.el7.x86_64 @base
Updated libdb-5.3.21-24.el7.x86_64 ?
Update 5.3.21-25.el7.x86_64 @base
Updated libdb-utils-5.3.21-24.el7.x86_64 ?
Update 5.3.21-25.el7.x86_64 @base
Updated libgcc-4.8.5-36.el7_6.2.x86_64 ?
Update 4.8.5-39.el7.x86_64 @base
Updated libgomp-4.8.5-36.el7_6.2.x86_64 ?
Update 4.8.5-39.el7.x86_64 @base
Updated libmount-2.23.2-59.el7_6.1.x86_64 ?
Update 2.23.2-61.el7.x86_64 @base
Updated libndp-1.2-7.el7.x86_64 ?
Update 1.2-9.el7.x86_64 @base
Updated libsmartcols-2.23.2-59.el7_6.1.x86_64 ?
Update 2.23.2-61.el7.x86_64 @base
Updated libss-1.42.9-13.el7.x86_64 ?
Update 1.42.9-16.el7.x86_64 @base
Updated libssh2-1.4.3-12.el7_6.3.x86_64 ?
Update 1.8.0-3.el7.x86_64 @base
Updated libstdc++-4.8.5-36.el7_6.2.x86_64 ?
Update 4.8.5-39.el7.x86_64 @base
Updated libteam-1.27-6.el7_6.1.x86_64 ?
Update 1.27-9.el7.x86_64 @base
Updated libtirpc-0.2.4-0.15.el7.x86_64 ?
Update 0.2.4-0.16.el7.x86_64 @base
Updated libuuid-2.23.2-59.el7_6.1.x86_64 ?
Update 2.23.2-61.el7.x86_64 @base
Dep-Install linux-firmware-20190429-72.gitddde598.el7.noarch @base
Updated lshw-B.02.18-12.el7.x86_64 ?
Update B.02.18-13.el7.x86_64 @base
Updated lz4-1.7.5-2.el7.x86_64 ?
Update 1.7.5-3.el7.x86_64 @base
Updated make-1:3.82-23.el7.x86_64 ?
Update 1:3.82-24.el7.x86_64 @base
Updated mariadb-libs-1:5.5.60-1.el7_5.x86_64 ?
Update 1:5.5.64-1.el7.x86_64 @base
Updated microcode_ctl-2:2.1-47.5.el7_6.x86_64 ?
Update 2:2.1-53.el7.x86_64 @base
Updated net-tools-2.0-0.24.20131004git.el7.x86_64 ?
Update 2.0-0.25.20131004git.el7.x86_64 @base
Updated nfs-utils-1:1.3.0-0.61.el7.x86_64 ?
Update 1:1.3.0-0.65.el7.x86_64 @base
Updated nmap-ncat-2:6.40-16.el7.x86_64 @base
Update 2:6.40-19.el7.x86_64 @base
Updated nspr-4.19.0-1.el7_5.x86_64 ?
Update 4.21.0-1.el7.x86_64 @base
Updated nss-3.36.0-7.1.el7_6.x86_64 ?
Update 3.44.0-4.el7.x86_64 @base
Updated nss-pem-1.0.3-5.el7_6.1.x86_64 ?
Update 1.0.3-7.el7.x86_64 @base
Updated nss-softokn-3.36.0-5.el7_5.x86_64 ?
Update 3.44.0-5.el7.x86_64 @base
Updated nss-softokn-freebl-3.36.0-5.el7_5.x86_64 ?
Update 3.44.0-5.el7.x86_64 @base
Updated nss-sysinit-3.36.0-7.1.el7_6.x86_64 ?
Update 3.44.0-4.el7.x86_64 @base
Updated nss-tools-3.36.0-7.1.el7_6.x86_64 ?
Update 3.44.0-4.el7.x86_64 @base
Updated nss-util-3.36.0-1.1.el7_6.x86_64 ?
Update 3.44.0-3.el7.x86_64 @base
Updated numactl-libs-2.0.9-7.el7.x86_64 ?
Update 2.0.12-3.el7.x86_64 @base
Updated openssh-7.4p1-16.el7.x86_64 ?
Update 7.4p1-21.el7.x86_64 @base
Updated openssh-clients-7.4p1-16.el7.x86_64 ?
Update 7.4p1-21.el7.x86_64 @base
Updated openssh-server-7.4p1-16.el7.x86_64 ?
Update 7.4p1-21.el7.x86_64 @base
Updated openssl-1:1.0.2k-16.el7_6.1.x86_64 ?
Update 1:1.0.2k-19.el7.x86_64 @base
Updated openssl-libs-1:1.0.2k-16.el7_6.1.x86_64 ?
Update 1:1.0.2k-19.el7.x86_64 @base
Updated parted-3.1-29.el7.x86_64 ?
Update 3.1-31.el7.x86_64 @base
Updated passwd-0.79-4.el7.x86_64 ?
Update 0.79-5.el7.x86_64 @base
Updated policycoreutils-2.5-29.el7_6.1.x86_64 ?
Update 2.5-33.el7.x86_64 @base
Updated policycoreutils-python-2.5-29.el7_6.1.x86_64 ?
Update 2.5-33.el7.x86_64 @base
Updated polkit-0.112-18.el7_6.1.x86_64 ?
Update 0.112-22.el7_7.1.x86_64 @updates
Updated procps-ng-3.3.10-23.el7.x86_64 ?
Update 3.3.10-26.el7.x86_64 @base
Updated python-2.7.5-80.el7_6.x86_64 ?
Update 2.7.5-86.el7.x86_64 @base
Updated python-chardet-2.2.1-1.el7_1.noarch ?
Update 2.2.1-3.el7.noarch @base
Updated python-jinja2-2.7.2-3.el7_6.noarch ?
Update 2.7.2-4.el7.noarch @base
Updated python-libs-2.7.5-80.el7_6.x86_64 ?
Update 2.7.5-86.el7.x86_64 @base
Updated python-linux-procfs-0.4.9-4.el7.noarch ?
Update 0.4.11-4.el7.noarch @base
Updated python-perf-3.10.0-957.27.2.el7.x86_64 ?
Update 3.10.0-1062.1.1.el7.x86_64 @updates
Updated python-requests-2.6.0-1.el7_1.noarch ?
Update 2.6.0-5.el7.noarch @base
Updated python-urllib3-1.10.2-5.el7.noarch installed
Update 1.10.2-7.el7.noarch @?base
Updated qemu-guest-agent-10:2.12.0-2.el7.x86_64 ?
Update 10:2.12.0-3.el7.x86_64 @base
Updated quota-1:4.01-17.el7.x86_64 ?
Update 1:4.01-19.el7.x86_64 @base
Updated quota-nls-1:4.01-17.el7.noarch ?
Update 1:4.01-19.el7.noarch @base
Updated readline-6.2-10.el7.x86_64 ?
Update 6.2-11.el7.x86_64 @base
Updated rpcbind-0.2.0-47.el7.x86_64 ?
Update 0.2.0-48.el7.x86_64 @base
Updated rpm-4.11.3-35.el7.x86_64 ?
Update 4.11.3-40.el7.x86_64 @base
Updated rpm-build-libs-4.11.3-35.el7.x86_64 ?
Update 4.11.3-40.el7.x86_64 @base
Updated rpm-libs-4.11.3-35.el7.x86_64 ?
Update 4.11.3-40.el7.x86_64 @base
Updated rpm-python-4.11.3-35.el7.x86_64 ?
Update 4.11.3-40.el7.x86_64 @base
Updated rsyslog-8.24.0-34.el7.x86_64 ?
Update 8.24.0-41.el7_7.x86_64 @updates
Updated selinux-policy-3.13.1-229.el7_6.15.noarch ?
Update 3.13.1-252.el7.1.noarch @updates
Updated selinux-policy-targeted-3.13.1-229.el7_6.15.noarch ?
Update 3.13.1-252.el7.1.noarch @updates
Updated sg3_utils-1.37-17.el7.x86_64 ?
Update 1.37-18.el7_7.1.x86_64 @updates
Updated sg3_utils-libs-1.37-17.el7.x86_64 ?
Update 1.37-18.el7_7.1.x86_64 @updates
Updated shadow-utils-2:4.1.5.1-25.el7_6.1.x86_64 ?
Update 2:4.6-5.el7.x86_64 @base
Updated sudo-1.8.23-3.el7.x86_64 ?
Update 1.8.23-4.el7.x86_64 @base
Updated systemd-219-62.el7_6.9.x86_64 ?
Update 219-67.el7_7.1.x86_64 @updates
Updated systemd-libs-219-62.el7_6.9.x86_64 ?
Update 219-67.el7_7.1.x86_64 @updates
Updated systemd-sysv-219-62.el7_6.9.x86_64 ?
Update 219-67.el7_7.1.x86_64 @updates
Updated teamd-1.27-6.el7_6.1.x86_64 ?
Update 1.27-9.el7.x86_64 @base
Updated tuned-2.10.0-6.el7_6.4.noarch ?
Update 2.11.0-5.el7_7.1.noarch @updates
Updated util-linux-2.23.2-59.el7_6.1.x86_64 ?
Update 2.23.2-61.el7.x86_64 @base
Updated vim-minimal-2:7.4.160-6.el7_6.x86_64 ?
Update 2:7.4.629-6.el7.x86_64 @base
Updated xfsprogs-4.5.0-19.el7_6.x86_64 ?
Update 4.5.0-20.el7.x86_64 @base
Updated yum-3.4.3-161.el7.centos.noarch ?
Update 3.4.3-163.el7.centos.noarch @base
Updated yum-plugin-fastestmirror-1.1.31-50.el7.noarch ?
Update 1.1.31-52.el7.noarch @base
Updated yum-utils-1.1.31-50.el7.noarch ?
Update 1.1.31-52.el7.noarch @base
Docker hub added pull limits so it would be nice to have an alternative registry to pull from
Should add an option to allow user to include arbitrary tar options. This would be useful for adding the ability to do things like incremental backups.
First of all, thanks for creating this image! It is really useful to me
With GitHub Actions, you can schedule automatic monthly builds of your Docker image to ensure that it's always updated with the latest Alpine version and the latest packages. To enable that, you just need to add the following schedule
section to your .github/workflows/publish-image.yml
and that's all.
on:
push:
branches:
- master
schedule:
# Runs automatically on the 27th of every month at 05.30 UTC
- cron: "30 05 27 * *"
I already use this feature (e.g. here) and it is super useful 🙂 thanks GitHub!
Let me know what you think about it and thanks again for your work :)
I really like loomchild/volume-backup
because it just gets the job done. The only thing I usually stumble accross is the bzip2 default. In my cases, it just eats lots of CPU time without any real benefit. If I do not backup to a file, but copy a volume to another host (on the same 1GBit network), I usually look at htop
asking myself why I did not pass -c none
.
I wondered if there is any adaptive compression algorithm around which automagically knows whether it should compress --fast
or --best
in order to provide maximum throughput. And it seems that Zstandard actually implemented this: facebook/zstd#75
As @loomchild mentioned in other issues, it's unlikely something other than bzip2 will be the default due to backwards compatibility. If you ever consider to change it, Zstandard with --adapt
might actually be a good choice.
For the time being, it might be a nice feature to pass the --adapt
argument to zstd
, if possible. (Notice: I didn't do any tests; I just assume --adapt
works as nice as described).
Having forked your repo - wondering what license it is under. Mind adding one?
Ive upgraded to debian11 and backups of images are now slow.
I dont know if its caused by #45 or something else. But zstd doesnt run with multithread option -T0
Tried to
env ZSTD_NBTHREADS=$(nproc) docker run --log-driver none -v $i:/volume --rm loomchild/volume-backup backup -c zstd > $BACKUP_DIR/volume-$i-$DATE.tar.zst
Not all cores used. Seems only one is running.
Tried to pass with -x '-T0'
Getting
tar: 0: Cannot stat: No such file or directory
tar: Error is not recoverable: exiting now
Probably function workaround now for me is:
docker run --log-driver none -v $i:/volume --rm loomchild/volume-backup backup -c 0 | zstd -T0 > $BACKUP_DIR/volume-$i-$DATE.tar.zst
Method to reproduce:
Backup:
Restore:
cat /var/hda/files/drives/drive12/downloads/Backups/dockerbackup/Tautulli.tar | docker run -i -v Tautulli:/volume --rm loomchild/volume-backup restore -
tar: invalid magic
tar: short read
Not quite sure what I'm doing wrong here!
Client:
Version: 18.09.0
API version: 1.39
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:48:52 2018
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.0
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:19:08 2018
OS/Arch: linux/amd64
Experimental: false
is there any reason this couldn't be run from cron on a regular basis for an automatic backup?
Hi,
When I create the tar.bz2 with this algorithm, it gives error when restoring.
I have tried 2 Linux and same error.
bzip2: invalid magic tar: Child died with signal 13 tar: Error is not recoverable: exiting now
Hello,
How would you suggest backup of multiple mounts on one container? I have one container with 8 mounts to host.
Thanks.
Would be awesome to provide the container with AWS credentials and a bucket and have it upload backups or pull them for restore.
Perhaps switch to pigz by default (but backwards-incompatible, maybe no problem).
Maybe install pbzip2 or lbzip2 for bzip.
It would be great if multiple arches were available in the package. I forked and built my own here: akutruff/volume-backup
I built on WSL2 using buildx. First I created a builder, and boostrapped it:
docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx inspect --bootstrap
Then I built it:
docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t akutruff/volume-backup:latest --push .
Tested on arm64.
Side note: This is an amazing little utility! Very clever implementation and nicely flexible. Good work!
The Readme suggests using cat archive | docker ... -
to restore backups, but that is another process call, wouldn't it be more efficient to just use docker ... - <archive
instead?
While it does work (I only tested with small backups) I'm not quite sure myself if there are major downsides to it.
When restoring a volume backup, its associated Labels are lost.
This would simplify options (no -, no compression). Need to make sure it works on Windows if want to support it.
Hi, would be great if script automatically stop all containers and run the backup and restart the containers?
Thank you
Remind me why we can't just back up or restore to folder
/var/lib/docker/volumes/<volume-name>
?
For backwards compatibility it would be nice to preserve the old syntax with two volumes in addition to this one.
From Holger's comment on my blog:
Very helpful, thank you. I felt inspired to avoid the dependency for mounting a host backup directory, by writing the archive content directly to stdout.
docker run –rm -v named-vol:/volume alpine tar -cjf – /volume > /tmp/backup.tar.bz2
Because my backup is not stored on the docker host directly:
docker run –rm -v named-vol:/volume alpine tar -cjf – /volume | ssh user@backuphost ‘cat > /tmp/backup.tar.bz2’
Hello, im using docker volume-backup for backup some volumes from sentry-on-premise.
I stop all containers and then backup containers with that loop
for i in sentry-data sentry-redis sentry-zookeeper sentry-kafka sentry-clickhouse sentry-symbolicator; do
docker run -v $i:/volume --rm loomchild/volume-backup backup -c zstd - > $BACKUP_DIR/volume-$i-$DATENOW.tar.zst
.
.
.
Backup running ok, but needs 2x space to make backup. Expecting when im packing to stdout not using too much space :( Is that bug or i need 2x space when doing backup ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.