kenkundert / emborg Goto Github PK
View Code? Open in Web Editor NEWInteractive command line interface to Borg Backup
License: GNU General Public License v3.0
Interactive command line interface to Borg Backup
License: GNU General Public License v3.0
I would like to pass the '--progress' option to Borg when running emborg create
. This will allow me to monitor the progress of an Enborg backup (which I usually run interactively.) It would also be cool to have a show_progress
setting in the config file allowing the user to configure Emborg to always show progress. Basically, I would like the same modification that was done in #28, but for the '--progress' options.
You can find out more about Borg's progress option here. (I've also quoted the text below.)
The --progress option shows (from left to right) Original, Compressed and Deduplicated (O, C and D, respectively), then the Number of files (N) processed so far, followed by the currently processed path.
when doing emborg prune
, the comments returned for prune are wrong
It would be nice to have a command to check an Emborg configuration for gross errors that are hard to detect without watching a backup (with -v) while it is running. Detecting cases where the src_dirs are empty (could indicate an unmounted source), where exclusions can't match any path specified in the src_dirs, or where exclusions exclude every file, and so forth, would reduce the risk of mistakes leading to unusable backups or wasted storage space.
If several hostnames are saved in the same repository starting their archive names with {host_name}-
, then for pruning, it is necessary to add {host_name}-
as a prefix with the --prefix
option.
See in Borg quickstart, the prefix is said to be "very important". Using prune without prefix in emborg, I inadvertently pruned other hostname's archives.
Then, one could argue it is more efficient to create one repository per server:
However this prevents deduplication across servers and when space is the priority having one single repo makes sense.
Some notes for the implementation:
emborg prune
, it results in the pruning of the repo without filtering.--prefix
is used for pruning and listing, having a general prefix option in the repo settings that applies to both will prevent listing any other archives, which can be detrimental. In this case we could offer a --ignore-prefix
option that ignores the prefix provided in the setting, it could apply to list and even prune: emborg --ignore-prefix list
will list all archives ignoring the prefix in settings.prune_prefix
and a list_prefix
option, this makes it a bit more complicated but addresses the points.I would go for 2) which sticks as much as possible to Borg naming of options and logic.
#19 is a first step for implementing 2) which should not create any issue, the next step could be to create the '--ignore-prefix' and extend the option to list as well. However this simple option addition already reaches the goal for safe pruning.
Borgmatic has a really nice feature where it will do a POST to healthchecks.io upon job completion. The POST body contains the stats of the job, not just the exit value. It would be nice if Emborg had this ability as well.
I have the following settings
:
root@raspberrypi4:/home/pi/.config/emborg# cat settings
# These settings are common to all configurations
# configurations
configurations = 'docker'
default_configuration = 'docker'
# encryption
encryption = 'keyfile' # borg encryption method
# Common choices are 'repokey' and 'keyfile'.
# With 'repokey' the encryption key is copied into repository, use this
# only if the remote repository is owned by you and is secure.
# With 'keyfile' the encryption key is only stored locally. Be sure to
# export it and save a copy in a safe place, otherwise you may not be
# able to access your backups if you lose your disk.
# specify either passphrase or avendesora_account
passphrase = 'xxx' # passphrase for encryption key
avendesora_account = '<<account-name>>' # avendesora account holding passphrase
# basic settings
# specify notify if batch and notifier if interactive
notify = '[email protected]' # who to notify when things go wrong
notifier = 'notify-send -u normal {prog_name} "{msg}"'
# interactive notifier program
remote_ratelimit = 2000 # bandwidth limit in kbps
prune_after_create = True # automatically run prune after a backup
check_after_create = 'latest' # automatically run check after a backup
# repository settings
repository = 'xxx:/home/xxx/borg_backups/{host_name}-{user_name}-{config_name}'
prefix = '{host_name}-'
# These may contain {<name>} where <name> is any of host_name, user_name,
# prog_name config_name, or any of the user specified settings.
# Double up the braces to specify parameters that should be interpreted
# directly by borg, such as {{now}}.
compression = 'lz4'
run_before_backup = 'service docker stop'
run_after_backup = 'service docker start'
healthchecks_uuid = 'xxx-xxx-xxx-xxx-xxx'
# filter settings
exclude_if_present = '.nobackup'
one_file_system = True
exclude_caches = True
# prune settings
keep_within = '1d' # keep all archives created in interval
keep_hourly = 48 # number of hourly archives to keep
keep_daily = 14 # number of daily archives to keep
keep_weekly = 8 # number of weekly archives to keep
keep_monthly = 12 # number of monthly archives to keep
keep_yearly = 1 # number of yearly archives to keep
When I run emborg create
, my docker service is not stopped while the backup takes place (I checked from another terminal). I assume the run_after_backup
is not run neither.
Also, when I emborg settings
I see:
root@raspberrypi4:/home/pi/.config/emborg# emborg settings
archive: '{host_name}-{{now}}'
'raspberrypi4-{now}'
avendesora_account: '<<account-name>>'
check_after_create: 'latest'
compression: 'lz4'
config_dir: '/root/.config/emborg'
config_name: 'docker'
configurations: 'docker'
default_configuration: 'docker'
encryption: 'keyfile'
exclude_caches: True
exclude_if_present: '.nobackup'
excludes: """
**/*~
**/__pycache__
**/*.pyc
**/.*.swp
**/.*.swo
"""
home_dir: '/root'
keep_daily: 14
keep_hourly: 48
keep_monthly: 12
keep_weekly: 8
keep_within: '1d'
keep_yearly: 1
log_dir: '/root/.local/share/emborg'
notifier: 'notify-send -u normal {prog_name} "{msg}"'
notify: '[email protected]'
one_file_system: True
passphrase: '<set>'
prefix: '{host_name}-'
'raspberrypi4-'
prune_after_create: True
remote_ratelimit: 2000
repository: 'xxx:/home/xxx/borg_backups/{host_name}-{user_name}-{config_name}'
'xxx:/home/xxx/borg_backups/raspberrypi4-root-docker'
src_dirs: """
/var/lib/docker/volumes/xxx1
/var/lib/docker/volumes/xxx2
/var/lib/docker/volumes/xxx3
"""
Same issue with the healthchecks_uuid
config. There is no ping taking place!
emborg is run as the root
user (so I can read the root owned docker volumes)
What am I doing wrong and why does emborg does not "see" my settings?
I attempted to backup my computer this morning with the emborg create
command, but it failed.
Judging from the error messages (see below), it seems to be an issue with borg on rsync.net's servers which they updated this morning. I'm not sure if there is much you can do from emborg to fix it, but I thought I should let others know. (Or you can correct me if I'm wrong and there is an issue with Emborg)
Here (log.txt) is the output of the create command with the verbose and narrate options. I have removed some things indicated by <<< >>>. The TypeError complaining about the number of arguments has always occurred in the past during successful runs.
There is no way to use the {health} format specifier when using the 'manifest' command in emborg.
It would be useful if this could be added.
I'm not sure how many active users emborg has, but have you considered creating an IRC channel for people to discuss it and get help? I think it'd be a good idea. Librera seems to be the IRC network of choice these days.
There's more info on registering a channel here:
I imagine it's a relatively straightforward process.
Hello! A small question on restoring from backups. I have several computers from which I backup to my server. In the emborg settings on computers, I specified encryption: 'repokey' and the passphrase. Suppose that one of the computers is completely out of order, i.e. the emborg settings were lost, but I remember the passphrase. How can I access computer backups on the server where the repositories are located?
If borg completes with a warning (say a file disappeared during the backup, so could not be processed) it appears that the healthcheck 'finished' ping isn't sent.
Borg's return codes are documented here:
https://borgbackup.readthedocs.io/en/stable/usage/general.html#return-codes
It looks like the 'success' ping is only sent if borg returns zero:
Lines 60 to 66 in 87d1624
Typo in the title of the release. Thought I'd point it out!
In my existing borg backup scripts, I like to see the results of a prune. To achieve this, I pass '--list' to the prune command.
Can this option be added to emborg prune also? It's available as an option to create.
Maybe using cookiecutter
Firstly, thanks for releasing Emborg!
The documentation at https://github.com/KenKundert/emborg/blob/c9cc1ace8758ad334bb929ae92ed98ee62988c78/doc/index.rst states the following:
repository: This is the location where all of your files are backed up to. It may be on a local file system or it may be remote, in which case it is accessed using ssh.
Which implies SSH is not used for repositories on the local filesystem. However, if I create a repository of the form: repository = 'mylocalhostname:/backup:/{host_name}-{user_name}-{config_name}'
it still tries to connect over SSH:
/backup $ emborg init
The authenticity of host 'mylocalhostname (::1)' can't be established.
ECDSA key fingerprint is SHA256:XXXXXXXXXXXXXXXXXXXXXXXXXXX.
Are you sure you want to continue connecting (yes/no)?
What's the correct way to tell Emborg to use a local repository, so that it doesn't use SSH? Is there something else that needs to be put in the first field (n.b. "localhost" yields the same problem)?
Depending on making a successful SSH connection via the loopback interface adds a superfluous dependency on SSH, and will break if a user does anything with SSH locally (changes the host keys, disables it, etc). If there's a way to avoid this, that would be great!
I'm trying to upgrade the Alpine package to the latest version, and add in the ability to run the tests during the package build.
I'm finding that a number of the tests are failing due to the way the Alpine CI process works (doing the build inside a Docker container, hence meaning the borg mount tests will fail as FUSE isn't available).
You may be able to see the output of a build here: https://gitlab.alpinelinux.org/adhawkins/aports/-/jobs/473511
Is there any way to skip selected tests with the test system you're using?
I currently run my emborg jobs individually from a script, that does a 'create' and 'prune' for each job, and then emails the results of that individual job before moving on to the next one.
Sometimes I want to exclude some jobs from the backup run. Currently the only way I can do that is to move the file out of the config directory.
It would be helpful if each individual job's settings file had an entry along the lines of 'disable: false / true', which would just (say) print out a message saying 'This job is disabled' if certain operations (say create, check and prune) were run on it.
Would that be useful generally do you think?
Would you consider adding an option for borg check --repair
as detailed here?
Maybe by using emborg check --repair
or even emborg repair
.
Thank you!
I realize this is probably more of a forum-question kinda thing, but I couldn't find one, so here goes nothing:
I created a repo using emborg
and uploaded some test data to it (rsync.net account). I was warned to extract my key, so I tried doing that:
pi@raspberrypi:~/.config/emborg $ emborg borg key export [email protected]:borg_rpi borg.ke
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4455, in main
exit_code = archiver.run(args)
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4387, in run
return set_ec(func(args))
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 154, in wrapper
return method(self, args, repository=repository, **kwargs)
File "/usr/lib/python3/dist-packages/borg/archiver.py", line 358, in do_key_export
manager.export(args.path)
File "/usr/lib/python3/dist-packages/borg/crypto/keymanager.py", line 85, in export
self.store_keyfile(path)
File "/usr/lib/python3/dist-packages/borg/crypto/keymanager.py", line 81, in store_keyfile
with open(target, 'w') as fd:
PermissionError: [Errno 13] Permission denied: 'borg.ke'
Platform: Linux raspberrypi 5.4.79-v7l+ #1373 SMP Mon Nov 23 13:27:40 GMT 2020 armv7l
Linux: debian 10.9
Borg: 1.1.9 Python: CPython 3.7.3
PID: 3949 CWD: /
sys.argv: ['/usr/bin/borg', '--remote-path', 'borg1', 'key', 'export', '[email protected]:borg_rp
i', 'borg.ke']
SSH_ORIGINAL_COMMAND: None
emborg error: unexpected exit status (2)
It seems the process CWDs to ´/´ and then has a hard time creating a file there. Does that mean that I have to use sudo
when extracting the key?
emborg info
shows me where the keyfile is saved: /home/pi/.config/borg/keys/...
, can't I just copy the keyfile from there?
Thanks for your help!
Trying to package up 1.29 for alpine, and running in to the following error when running the tests:
____________________ ERROR collecting tests/test_emborg.py _____________________
ImportError while importing test module '/home/builder/aports/community/emborg/src/emborg-1.29/tests/test_emborg.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/test_emborg.py:15: in <module>
from shlib import Run, cd, cp, cwd, ln, lsf, mkdir, rm, set_prefs, touch
/usr/lib/python3.10/site-packages/shlib/__init__.py:5: in <module>
from .shlib import (
E ImportError: cannot import name 'brace_expand' from 'shlib.shlib' (/usr/lib/python3.10/site-packages/shlib/shlib.py)
=========================== short test summary info ============================
I can successfully build and test 1.28, so am pretty sure that my build is Ok. There don't seem to be any new dependencies in emborg, and shlib doesn't seem to have changed since the release that's being installed in alpine.
Any suggestions?
Hello!
The emborg-overdue does not send letters after completion. The logs do not even show an attempt to send a letter. At the same time, an ordinary emborg always sends letters successfully.
Config file:
default_maintainer = 'backup@localhost'
dumper = 'backup@localhost'
default_max_age = 25 # hours
root = '/mnt/backup_data/borg'
repositories = [
dict(host='fs (/)', path='fs/root'),
]
Messages for backup@localhost are forwarded to Gmail.
Emborg-overdue output:
HOST: fs (/)
sentinel file: /mnt/backup_data/borg/fs/root/index.43
last modified: 2019-08-07T09:41:23.706817+00:00
since last change: 120.9 hours
maximum age: 25 hours
overdue: True
I found out that while you have an archive mounted you get errors while trying to create new backups:
➜ emborg -c linked_path create
emborg error: borg create:
Failed to create/acquire the lock /home/stratos/borg_backups2/serenity-stratos-linked_path/lock (timeout).
The message was confusing enough and it took me QUITE some time until I figured out that it might be due to the currently mounted archive. Once I emborg unmounted the archive the timeouts were gone.
Can this particular case be captured, and maybe a more informative message shown? Maybe even something "Do you currently have any archives mounted?".
There is a new --latest
option for mounting the latest archive, however by default other commands like extract
and manifest
apply to the latest archive while mount
by default mounts all archives.
Therefore wouldn't it make more sense that mount by default mounts the latest archive and that mount --all
mounts all archives?
Just a question as I may miss some logic behind.
Related to #21 (comment)
CREATE BACKUP WITH PORT 2222 WITH BORG
borg --verbose init --encryption=none test
sudo borg create --list --show-rc --verbose --progress --stats --comment "Backup Now." ssh://[email protected]:2222/test::Backup-{now:%Y-%m-%dT%H:%M:%S.%f} /home/test
INFO ON BACKUP USING EMBORG OR ANY COMMAND
Note:Using ssh_command = "ssh -p 2222"
emborg info
Warning: The repository at location ssh://[email protected]/test was previously located at ssh://[email protected]:2222/test
Do you want to continue? [yN] Aborting.
Repository access aborted
The Emborg Repository location won't include the port and borg will see it as a different location.
I tried:
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no
with limited success and would prefer not to if there is a better option.
Hello,
I've been trying out emborg as a frontend for borg.
But it seems like most examples listed in the docs are not working (making it difficult to see archives, restore etc)
I ran the following emborg commands:
$ emborg -c test init # created the remote borg repo
$ emborg -c test create
Running Borg create command ...
Checking archive ...
emborg error: no archives are available.
This error occurred while checking the archives.
No error was reported while creating the archive.
# emborg can't find the archive created?
$ emborg -c test list
# but borg can?
$ emborg -c test borg list @repo
serv-2021-03-07T17:23:162021-03-07T17:23:16 Sun, 2021-03-07 17:23:17 [9f42bb5ff2affc23e7ac21dd8550cd2762043e3d27bcd1e113317f05214b247f]
serv-2021-03-07T17:23:522021-03-07T17:23:52 Sun, 2021-03-07 17:23:53 [8bbab3b27eab798d458b4b9aef9aec6e2a8baf69a375f0e107ad77ae91083a76]
serv-2021-03-07T17:28:552021-03-07T17:28:55 Sun, 2021-03-07 17:28:56 [560e3a7c630f51b0298543728b3f246899b12d6af914b13e69e4705c538fd255]
serv-2021-03-07T17:49:232021-03-07T17:49:23 Sun, 2021-03-07 17:49:24 [5810e276481b1020189a0c7218543f203a43d55fad4063f1070c0862bcc6acee]
$ mv test_dir test_dir_2
# because emborg can't find the archive I can't restore it :/
$ emborg -c test restore test_dir
emborg error: no archives are available.
$ emborg version
emborg version: 1.20.0 (2021-02-13) [Python 3.9.2].
It seems that emborg can't find the archives?
Even though emborg created them and that borg can find the archives.
The test configuration is stored in ~/.config/emborg/test
# Settings for a test configuration
# use of absolute paths is recommended
src_dirs = '/home/zeus/vpn' # paths to directories to be backed up
The general settings file is stored in ~/.config/emborg/settings
And looks as followed (omitting sensitive data)
# These settings are common to all configurations
# configurations
configurations = 'home root test'
default_configuration = 'home'
# encryption
encryption = 'repokey' # borg encryption method
# Common choices are 'repokey' and 'keyfile'.
# With 'repokey' the encryption key is copied into repository, use this
# only if the remote repository is owned by you and is secure.
# With 'keyfile' the encryption key is only stored locally. Be sure to
# export it and save a copy in a safe place, otherwise you may not be
# able to access your backups if you lose your disk.
# specify either passphrase or avendesora_account
passphrase = '<my_passphrase>' # passphrase for encryption key
# basic settings
# specify notify if batch and notifier if interactive
notify = '<my_mail>' # who to notify when things go wrong
notifier = 'notify-send -u normal {prog_name} "{msg}"'
# interactive notifier program
remote_ratelimit = 2000 # bandwidth limit in kbps
prune_after_create = True # automatically run prune after a backup
check_after_create = 'latest' # automatically run check after a backup
show_progress = True
# repository settings
repository = 'backup@<server>:/home/backup/Sync/borg/{host_name}-{user_name}-{config_name}'
prefix = '{host_name}-{{now}}'
compression = 'zstd'
# filter settings
exclude_if_present = '.nobackup'
Any help would be greatly appreciated.
PS: Running any option with the --verbose
flag simply prints Running borg <command> command ...
and nothing more
I did an 'emborg manifest' on a raspberry pi running raspbian 10.3 (a derivative of debian 10.3) and saw the following traceback. The files were not listed by the manifest call.
File "/usr/local/bin/emborg", line 10, in <module> sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/emborg/main.py", line 96, in main cmd_name, args, settings, emborg_opts
File "/usr/local/lib/python3.7/dist-packages/emborg/command.py", line 162, in execute exit_status = cls.run(name, args if args else [], settings, options)
File "/usr/local/lib/python3.7/dist-packages/emborg/command.py", line 1127, in run sep="\n",
File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 1452, in __call__ INFORMER._report(args, kwargs, self)
File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 1936, in _report multiline, continuing, options
File "/usr/local/lib/python3.7/dist-packages/inform/inform.py", line 2028, in _show_msg
print(': '.join(cull([header, culprit, message])), **options)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0151' in position 82: ordinal not in range(256)
root@raspberrypi:~# client_loop: send disconnect: Broken pipe
With the addition of the hooks, emborg now depends on requests also. However, this dependency is not mentioned in setup.py
Currently emborg mount
accepts a --date
option where you can select a given date and mount the archive(s) found for that particular timeframe.
I think it would be useful to also have an option to mount the most recent archive found in the repository without having to explicitly give a date.
A possible suggestion would be --last
, --latest
, or something similar.
Thank you.
Before actually reading the docs for the diff
command, I was thinking/hoping that it would do a diff of the contents of the current dir against the latest backup, but alas it does not... :(
My use case, is that I want to easily see if the contents of the current dir have been changed since my last backup. I never thought that I will need the diffs between two arbitrary previous backups (although useful for some cases, I guess).
My suggestion would be that if you pass NO arguments to diff
then you compare the last backup with the current folder (and subfolders, I would guess)
If you pass a single parameter (of any previous backup) you can diff that backup with the current folder.
Passing two params is what you currently have.
As a workaround, for now I mount the latest backup in a random folder and I manually diff
the folders (having to laboriously copy the full path to the folders I need to compare). Doable, but I believe (as I originally thought) this functionality should be native to emborg.
BTW, while I was doing the above mounting exercise I came across [another issue (#48) (I believe)...
It seems that if I try to restore files from a linked folder I get an error.
A demonstration of the failure:
# First create the folder structures
root@yoda:/# cd /tmp
root@yoda:/tmp# mkdir -p folder/subfolder
root@yoda:/tmp# touch folder/subfolder/file.txt
root@yoda:/tmp# ln -s folder linked_folder
# Show the emborg configuration files
root@yoda:/tmp# cat ~/.config/emborg/plain_path
src_dirs = '/tmp/folder/subfolder' # absolute path to directory to be backed up
root@yoda:/tmp# cat ~/.config/emborg/linked_path
src_dirs = '/tmp/linked_folder/subfolder' # absolute path to directory to be backed up
# Create the two borg backups and show their manifests
root@yoda:/tmp# emborg -c plain_path create
root@yoda:/tmp# emborg -c linked_path create
root@yoda:/tmp# emborg -c plain_path manifest
Archive: yoda-2021-08-19T20:40:10
tmp/folder/subfolder
tmp/folder/subfolder/file.txt
root@yoda:/tmp# emborg -c linked_path manifest
Archive: yoda-2021-08-19T20:40:27
tmp/linked_folder/subfolder
tmp/linked_folder/subfolder/file.txt
# Perform restore on the original (unlinked) folder
root@yoda:/tmp# cd folder/subfolder
root@yoda:/tmp/folder/subfolder# ls -al
total 0
drwxr-xr-x 2 root root 60 Aug 19 20:39 .
drwxrwxrwt 14 root root 300 Aug 19 20:41 ..
-rw-r--r-- 1 root root 0 Aug 19 20:39 file.txt
root@yoda:/tmp/folder/subfolder# mv file.txt file.txt.deleted
root@yoda:/tmp/folder/subfolder# emborg -c plain_path restore file.txt
Archive: yoda-2021-08-19T20:40:10
root@yoda:/tmp/folder/subfolder# ls -al
total 0
drwxr-xr-x 2 root root 80 Aug 19 20:42 .
drwxrwxrwt 14 root root 300 Aug 19 20:42 ..
-rw-r--r-- 1 root root 0 Aug 19 20:39 file.txt
-rw-r--r-- 1 root root 0 Aug 19 20:39 file.txt.deleted
# Try to perform the same on the symlinked folder
root@yoda:/tmp/folder# cd ../..
root@yoda:/tmp# cd linked_folder/subfolder
root@yoda:/tmp/linked_folder/subfolder# ls -al
total 0
drwxr-xr-x 2 root root 80 Aug 19 20:42 .
drwxrwxrwt 14 root root 300 Aug 19 20:42 ..
-rw-r--r-- 1 root root 0 Aug 19 20:39 file.txt
-rw-r--r-- 1 root root 0 Aug 19 20:39 file.txt.deleted
root@yoda:/tmp/linked_folder/subfolder# rm file.txt
root@yoda:/tmp/linked_folder/subfolder# emborg -c linked_path restore file.txt
Archive: yoda-2021-08-19T20:40:27
emborg error: borg extract: Include pattern 'tmp/folder/subfolder/file.txt' never matched.
It seems like emborg is trying to match the original path (the path that the symbolic link is pointing too) instead of the actual path.
Isn't this a bug on emborg?
Thanks, btw for an otherwise cool utility! :)
It took me a while to figure out that the following error was refering to a problem with my avendesora config, not with emborg or borg itself.
$ emborg init
emborg warning: rsync: url attribute found, should probably be urls.
emborg error: Connection closed by remote host. Is borg working on the server?
Healthchecks.io can be self-hosted. It would be useful if emborg could allow the ping URL for healthchecks.io to be specified, allowing a self-hosted solution to be used.
Emborg 1.3.0
File "/home/foo/.local/lib/python3.5/site-packages/emborg/settings.py", line 112
f'Configuration directory created: {parent!s}.',
^
SyntaxError: invalid syntax
Hi, I need to add a folder which has spaces in its name to the pattern of folders to backup.
In example, how can I add ~/.config/My Dir/
to my pattern which i.e. is:
patterns = '''
R ~/Documents
R ~/Music
- ~/Documents/work
'''
I need to backup a folder which is located inside an excluded folder.
More precisely, I need to backup ~/.cache/evolution/
, while ~/.cache/
is excluded.
This is my current configuration
src_dirs = '''
~/
~/.cache/evolution/
/home/shared/
'''.split() # absolute path to directory to be backed up
excludes = '''
~/**/__pycache__
~/**/*.pyc
~/**/.*.swp
~/**/.*.swo
~/**/node_modules
~/.cache
~/.npm
~/.pnpm-store
~/.local/share/baloo
~/.local/share/Trash
~/.thumbnails
'''.split()
But it does not seem to work. Is it possible to do so? And how?
It would be nice to use also the --exclude-from
file option of Borg, this avoids repeating the same exclude paths in several setting files and should be relatively straightforward to implement.
I can help if needed.
Hi,
I can't seem to get the "passcommand" config directive to work as it does with plain Borg itself, but perhaps I am doing something incorrectly that is Emborg-specific, but wasn't sure if it was a bug?
passcommand = 'gpg -qd /home/user/.store-auth.gpg'
I've always had this same setup working with plain Borg itself, as well as with Restic and Rclone using these env vars:
BORG_PASSCOMMAND=gpg -qd /home/user/.store-auth.gpg
RCLONE_PASSWORD_COMMAND=gpg -qd /home/user/.store-auth.gpg
RESTIC_PASSWORD_COMMAND=gpg -qd /home/user/.store-auth.gpg
So it works fine with plain Borg and others, but not with Emborg (neither the "BORG_PASSCOMMAND" env var, nor the explicit Emborg "passcommand" config directive). Everything else with Emborg works fine, so long as I don't use any encryption. But with encryption, I have to use plain Borg directly.
I get this error with verbose enabled:
$ emborg -vc vault init
emborg error: Cannot determine the encryption passphrase.
The log largely says the same:
failure detected:
Cannot determine the encryption passphrase.
emborg error: Cannot determine the encryption passphrase.
emborg: terminates with status 2 on Tuesday, 17 March 2020 at 11:32:27 AM EDT.
I get the same behavior on both Linux (Manjaro) and MacOS (Catalina/10.15), both with the same Emborg/Borg versions:
$ emborg version
emborg version: 1.15.0 (2020-03-06) [Python 3.8.1].
$ borg --version
borg 1.1.10
Hi,
is there already a solution to backup to multiple repositories? E.g. BorgBase and another internal repository?
Kind regards,
Andreas
Possible enhancement is to have in configuration exclude_from
accepting list of filenames instead of a single name as Borg accepts several instances of exclude_from
.
I would like to automatically pass '--stats' to the prune function when I run 'emborg -v --narrate create' with prune_after_create enabled in the settings file. I like to see what emborg and borg are doing (though '--list' for prune might be too much info). I would guess that the easiest way might be to change the default behaviour of verbose to include stats for prune. But I don't know it that is true, how to do it myself, or if others would appreciate that default behaviour.
~/.config/emborg
is used to save log and last backup information, this should rather go in ~/.local/share/emborg
.
#20 implements this suggestion.
# emborg help
Traceback (most recent call last):
File "/usr/local/bin/emborg", line 7, in <module>
from emborg.main import main
File "/usr/local/lib/python3.5/dist-packages/emborg/__init__.py", line 4, in <module>
from .settings import Settings as Emborg
File "/usr/local/lib/python3.5/dist-packages/emborg/settings.py", line 112
f'Configuration directory created: {parent!s}.',
^
SyntaxError: invalid syntax
emborg was installed as pip3 install emborg
, not pip3 install --user emborg
because doing so, as root for install and as root to run, yield nothing.
Could this be because borg was installed as a single binary?
/usr/local/bin/borg -> /folder/where/borg-binary-is/borg-linux64
I am a great fan of emborg, I have created and maintain the Arch Linux package.
I have noticed the addition of an external dependency to quantiphy
in the latest version and have therefore created a new package for it.
I just wanted to highlight this makes the installation of emborg dependent to one more "unofficial" package. Not an issue at all, just for your information, dependencies to non-standard python packages is not neutral.
Thanks again for what you are offering with emborg
this issue can be closed as it does not require specific action, just fyi.
I often use the borg
command on the repo using the @repo
variable, however how can I run a borg command on a particular archive of the repo? I have tried several combinations like @repo::Monday
without success.
Is it possible to make optional email notifications upon successful completion of a backup?
Given multiple configurations on ~/.config/emborg/
, let's call them home
, lamp
, data
, and so on, I think it would be helpful to have an option to run all found configs with a single command.
So, instead of running emborg -c home && emborg -c lamp && emborg -c data
, maybe one could run emborg -a
or emborg --all
, or emborg
or something similar.
I think it would be helpful to have the ability to set the -v
, --verbose
options as default, maybe from withing the config files.
The default configuration files created by running emborg init are not able to interpret source directories or exclude masks that include spaces or single quotes. The Python split() method does not have provision for escaping spaces or other characters. As a result, including a path with a space in an Emborg configuration file will result in Emborg attempting to parse the path as two, incomplete, paths, with undesirable results.
If the configuration files are rewritten to use Python list syntax, rather than split(), Emborg processes paths with spaces correctly. For example, this configuration does work without any changes to Emborg proper:
src_dirs = ['/mnt/drive to back up', '/mnt/other drive to back up' ]
excludes = [
'*/desktop.ini',
'*/Thumbs.db',
'*/stuff to exclude/*'
]
Note however, that if there are any syntax errors in the exclusion list (such as a missed trailing comma), Emborg will execute a backup but ignore the entire exclusion list. Presumably this is also true if using the standard configuration files. I feel syntax errors in the configuration files should be treated as fatal errors.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.