yunohost-apps / archivist_ynh Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
We need to work an a new config panel config file and maybe on scripts too.
Help is appreciated :)
I would like to suggest an improvement to the log message we receive by email.
First of all, error messages such as "Not enough free space on…" could be highlighted (using bold ?) to catch them immediately when reading the email.
Secondly, this "Not enough free space on…" error message could show how big the archive was meant to be, and maybe how much space is available on the targeted storage.
That would help us diagnose this lack of space faster (and maybe detect any sudden drastic increase) to work on a workaround.
I guess it could be done by showing 1) Yunohost message about the final size of the backup 2) the result of df /storage/path
command.
I know how to implement the second improvement, but not the first one : how do you catch that log ?
Hi there, I'm wondering if there is some further documentation on using Archivist? It does not appear to be working, and I'm not sure whether I have done everything that I was supposed to do. I am assuming archivist will run when I generate a backup on YunoHost?
So far, I have taken the following steps:
## OPTIONS
# Backup directory.
backup_dir=/home/yunohost.app/archivist/backup
# Encrypted backup directory.
# Usually just next to the backup directory
enc_backup_dir=/home/yunohost.app/archivist/encrypted_backup
# Encryption
# true/false
encrypt=false
# Password for encryption
# This file should be set at chmod 400
cryptpass=na
# Compression algorithm to use
# gzip
# Standard compression algorithm.
# Fast and light on ressources.
# lzop
# Very fast algorithm.
# Low efficency
# zstd
# Zstandard algorithm
# Similar to gzip but faster
# bzip2
# Very slow but very effective.
# Light on ressources.
# lzma
# Faster than bzip2 and even more effective.
# But demanding in resources.
# lzip
# Similar to lzma.
# xz
# Similar to lzma but more recent.
#
# none
# No compression
# Concatenation in a tar file only
# symlink
# No compression
# Symbolic link to the YunoHost tar file from /home/yunohost.backup/archives
# The symbolic link is used only for the local $backup_dir of archivist.
# Real tar files are send to recipients.
#
# This mode allow archivist to not duplicate locally the backup if not compressed.
# WARNING: Does not work with encryption !
#
# Default: gzip
ynh_compression_mode=gzip
files_compression_mode=gzip
# --------------------------------------------------------------------------------------
## YUNOHOST BACKUPS
# YunoHost backups
# Specific backup made by 'yunohost backup' command
# ynh_core_backup=true/false: Make a backup of the core of YunoHost, without any apps.
ynh_core_backup=true
ynh_core_pre_backup=
ynh_core_post_backup=
# ynh_app_backup=APP: Make a backup of an app.
# You can add as many lines as you want to save multiple apps.
# List all apps with sudo yunohost app list --installed | grep "id: "
# ynh_app_backup=wordpress
# If you don't want an app with a lot of data to backup those big files, please add 'do_not_backup_data' after the app name.
# ynh_app_backup=transmission do_not_backup_data
ynh_app_pre_backup=
ynh_app_post_backup=
ynh_app_backup=alltube
ynh_app_backup=archivist
ynh_app_backup=cryptpad
ynh_app_backup=element
ynh_app_backup=encryptic
ynh_app_backup=gitea
ynh_app_backup=grocy
ynh_app_backup=lstu
ynh_app_backup=lufi
ynh_app_backup=piwigo
ynh_app_backup=rainloop
ynh_app_backup=restic
ynh_app_backup=searx
ynh_app_backup=strut
ynh_app_backup=synapse
ynh_app_backup=ttrss
ynh_app_backup=wallabag2
ynh_app_backup=zerobin
ynh_app_backup=
# --------------------------------------------------------------------------------------
## FILES BACKUPS
# Max size for each backup in Mb.
# Be carreful, it's only a soft limit, that means the script will try to limit each backup.
# Because, it will never made more than one backup for a single directory
# (Even if it's a real big directory without subdirectories).
# And the same if there's files next to subdirectories, only one backup will be made for all this files.
max_size=1000
# Backup of files and directories
# Add as many lines as you want for each directory or file you want to be backuped.
# Exclude a directory, a file, or multiples files with a regex with the exclude_backup instruction.
# file_to_backup="/directory/to backup"
# file_to_backup=/file/to backup
# exclude_backup="/directory/to exclude"
# exclude_backup=/file/to exclude
# exclude_backup="/exclude/all/tarball/in_this_directory/.tar.gz$"
files_pre_backup=
files_post_backup=
file_to_backup=
exclude_backup=
# --------------------------------------------------------------------------------------
## BACKUPS RECIPIENTS
# All options following a recipient name will be enabled for this recipient only.
# > recipient name=Name of this recipient.
# type=Choose between one of the scripts in the "senders" directory.
# destination directory=Directory where to put the backup in the other side.
# encrypt=Override main option for encryption. (true/false) (Optionnal option)
# specific option for type=See the chosen script to know what's the options.
# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)
# > recipient name=local example
# type=local
# destination directory=/my/local/backup
# encrypt=false
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
# > recipient name=rsync ssh example
# type=rsync_ssh
# destination directory=backup
# encrypt=false
# ssh_host=domain.tld
# ssh_user=user
# ssh_port=22
# ssh_key=/home/user/.ssh/id_rsa
# ssh_pwd=
# ssh_options=
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
> recipient name=[B2 NAME]
type=b2
destination directory=/
encrypt=false
b2_bucket=[BUCKET NAME]
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
YunoHost encountered an internal error when trying to access the config-panel pf archivist.
Config panel question 'encrypt' should be initialized with a value during install or upgrade.
Hello everyone,
when I try to backup my apps, archivist has this error when creating a temporary backup for the app seafile.
Traceback (most recent call last): │# type=Choose between one of the scripts in the "senders" directory.
File "/usr/bin/yunohost", line 72, in <module> │# destination directory=Directory where to put the backup in the other side.
parser=parser │# encrypt=Override main option for encryption. (true/false) (Optionnal option)
File "/usr/lib/moulinette/yunohost/__init__.py", line 29, in cli │# specific option for type=See the chosen script to know what's the options.
top_parser=parser │# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
File "/usr/lib/python2.7/dist-packages/moulinette/__init__.py", line 120, in cli │# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)
args, output_as=output_as, timeout=timeout │
File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/cli.py", line 477, in run │> recipient name=Fritzbox NAS
ret = self.actionsmap.process(args, timeout=timeout) │type=local
File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 592, in process │destination directory=/mnt/fritzbox/TimeMachine/yh-backups
return func(**arguments) │# encrypt=false
File "/usr/lib/moulinette/yunohost/backup.py", line 2062, in backup_create │# pre_backup=
backup_manager.backup() │# post_backup=
File "/usr/lib/moulinette/yunohost/backup.py", line 730, in backup │# exclude backup=/exclude/dir
method.mount_and_backup() │# include backup=
File "/usr/lib/moulinette/yunohost/backup.py", line 1574, in mount_and_backup │
self.backup() │# > recipient name=local example
File "/usr/lib/moulinette/yunohost/backup.py", line 1745, in backup │# type=local
shutil.copytree(source, dest) │# destination directory=/my/local/backup
File "/usr/lib/python2.7/shutil.py", line 231, in copytree │# encrypt=false
raise Error, errors │# pre_backup=
shutil.Error: [('/opt/yunohost/seafile/ccnet/ccnet.sock', '/home/admin/temp/ynh_backup/temp/apps/seafile/backup/opt/yunohost/se│Backup_list.conf 122,1 76%
afile/ccnet/ccnet.sock', "[Errno 6] No such device or address: '/opt/yunohost/seafile/ccnet/ccnet.sock'")] │
>>> The temporary backup failed..
In my Backup_list.conf I added a pre-backup action to stop both seafile services: seafile and seahub.
ynh_app_pre_backup=/etc/backups/services-stop.sh
ynh_app_post_backup=/etc/backups/services-start.sh
ynh_app_backup=archivist
ynh_app_backup=seafile do_not_backup_data
Backing up the seafile app via command line BACKUP_CORE_ONLY=1 yunohost backup create --app seafile
works flawlessly.
Any ideas or suggestions what is going wrong?
Problem Description:
The MongoDB service is shown as not found or inactive despite its proper functionality. When loading the configuration panel, an error message appears stating: “Config panel question ‘encrypt’ should be initialized with a value during install or upgrade.”
mey system https://paste.yunohost.org/raw/xihaziyavu
$ systemctl status mongod$
● mongod.service
Loaded: not-found (Reason: Unit mongod.service not found.)
Active: active (running) since Tue 2023-05-16 10:37:20 UTC; 5 days ago
Main PID: 1135325
CPU: 58min 36.292s
CGroup: /system.slice/mongod.service
└─1135325 /usr/bin/mongod --config /etc/mongod.conf
carrabelloy@srv:
https://forum.yunohost.org/t/unkown-status-for-mongodb/24837?u=carrabelloy
How to post a meaningful bug report
A clear and concise description of what the bug is.
yunohost app info $app_id
sudo yunohost app install the_app
A clear and concise description of what you expected to happen. You can remove this section if the command above is enough to understand your intent.
When an operation fails, YunoHost provides a simple way to share the logs.
yunohost log display [log name] --share
. If you missed it, you can find the log ID of a previous operation using yunohost log list
.After sharing the log, please copypaste directly the link provided by YunoHost (to help readability, no need to copypaste the entire content of the log here, just the link is enough...)
If applicable and useful, add screenshots to help explain your problem.
Hi there,
is there a way to disable the daily mails from cron to yh-root?
It may be a silly question, but how to restore these backups?
Hello,
I am wondering if an app can register something for the auto-diagnosis screen (@YunoHost-Apps/apps-group ?).
If so, could it be that this app would add a check in that diagnosis that would scan installed apps, and warn if automated backup is not set up for one of them.
This way, if you install a new app after archivist first config, you don't have to think about adding it to automated backups : the diagnosis will warn you have this app having no backup setup.
What do you think about such a feature ?
I am mentioning this issue YunoHost-Apps/invidious_ynh#17, here too. Might be relevant.
Hi,
I launched /opt/yunohost/archivist/archivist.sh
after removing my Laverna app, and without editing the conf file (yeah, I should have, I simply forgot). I already made a backup with archivist, some weeks ago.
Here is the output:
>> Make a temporary backup for laverna_backup
Error: App 'laverna' will not be saved
Error: There is nothing to save
rm: cannot remove ‘/home/yunohost.app/archivist/backup/ynh_backup/temp/info.json’: No such file or directory
find: `/home/yunohost.app/archivist/backup/ynh_backup/temp': No such file or directory
>>> This backup is different than the previous one
>>>> Make a real backup for laverna_backup
Error: App 'laverna' will not be saved
Error: There is nothing to save
cp: cannot stat ‘/home/yunohost.backup/archives/laverna_backup.tar.gz’: No such file or directory
cp: cannot stat ‘/home/yunohost.backup/archives/laverna_backup.info.json’: No such file or directory
1.7M /home/yunohost.app/archivist/backup/ynh_backup/laverna_backup.tar.gz
If "Make a temporary backup" fails, maybe we should not run "Make a real backup" ?
Hello,
I have these messages in my logs :
Erreur : Script de sauvegarde ‘conf_ssh’ inconnu
Erreur : Script de sauvegarde ‘conf_ssowat’ inconnu
Erreur : Script de sauvegarde ‘conf_ynh_firewall’ inconnu
Erreur : Script de sauvegarde ‘conf_xmpp’ inconnu
Erreur : Script de sauvegarde ‘conf_nginx’ inconnu
Erreur : Script de sauvegarde ‘conf_cron’ inconnu
Erreur : Script de sauvegarde ‘conf_ynh_currenthost’ inconnu
Not sure if this is an issue for the core backup.
6ril
Is it possible to add the possibility to use borg for the backups ?
When I try to open the experimental config panel on apps/archivist/config-panel, the server does not load nothing, keeps on loading.
"host": "Debian 9.11",
"kernel": "4.9.0-11-amd64",
"packages": {
"yunohost": {
"repo": "stable",
"version": "3.6.4.6"
},
"yunohost-admin": {
"repo": "stable",
"version": "3.6.4"
},
"moulinette": {
"repo": "stable",
"version": "3.6.4.1"
},
"ssowat": {
"repo": "stable",
"version": "3.6.4"
Problem: As Yunohost 4.1 is out now, backups are no longer compressed https://forum.yunohost.org/t/yunohost-4-1-release-sortie-de-yunohost-4-1/13893
We can still compress them afterwards, but it adds some (manual) extra work.
For regular automated backup it can lead to an important storage space use, or would require manual work to fix this (then using this app would no longer be as "install&forget" than it was before).
Proposal : Would it be possible to add an option to compress backups ?
I'm willing to help testing it :)
Benefits : this would save a lot of storage space, while not requiring users to regularly think about it and spare some time doing it manually (with CLI).
Additional proposal : use Zstandard compression instead of Gzip to reduce CPU usage and storage space (detailed proposal here https://forum.yunohost.org/t/backups-implementing-zstandard-compression-instead-of-gzip/13379)
Not sure why it doesn't transfer backup to SSH server via rsync.. here's my config from /opt/yunohost/archivist/Backup_list.conf
## OPTIONS
# Backup directory.
backup_dir=/home/yunohost.app/archivist/backup
# Encrypted backup directory.
# Usually just next to the backup directory
enc_backup_dir=/home/yunohost.app/archivist/encrypted_backup
# Encryption
# true/false
encrypt=true
# Password for encryption
# This file should be set at chmod 400
cryptpass=/opt/yunohost/archivist/passkey
# --------------------------------------------------------------------------------------
## YUNOHOST BACKUPS
# YunoHost backups
# Specific backup made by 'yunohost backup' command
# ynh_core_backup=true/false: Make a backup of the core of YunoHost, without any apps.
ynh_core_backup=true
ynh_core_pre_backup=
ynh_core_post_backup=
# ynh_app_backup=APP: Make a backup of an app.
# You can add as many lines as you want to save multiple apps.
# List all apps with sudo yunohost app list --installed | grep "id: "
# ynh_app_backup=wordpress
# If you don't want an app with a lot of data to backup those big files, please add 'do_not_backup_data' after the app name.
# ynh_app_backup=transmission do_not_backup_data
ynh_app_pre_backup=
ynh_app_post_backup=
ynh_app_backup=archivist
ynh_app_backup=portainer
ynh_app_backup=
# --------------------------------------------------------------------------------------
## FILES BACKUPS
# Max size for each backup in Mb.
# Be carreful, it's only a soft limit, that means the script will try to limit each backup.
# Because, it will never made more than one backup for a single directory (Even if it's a real big directory without subdirectories). And the same if there's files next to subdirectories, only one backup will be made for all this files.
max_size=500
# Backup of files and directories
# Add as many lines as you want for each directory or file you want to be backuped.
# Exclude a directory, a file, or multiples files with a regex with the exclude_backup instruction.
# file_to_backup="/directory/to backup"
# file_to_backup=/file/to backup
# exclude_backup="/directory/to exclude"
# exclude_backup=/file/to exclude
# exclude_backup="/exclude/all/tarball/in_this_directory/.tar.gz$"
files_pre_backup=
files_post_backup=
file_to_backup=
exclude_backup=
# --------------------------------------------------------------------------------------
## BACKUPS RECIPIENTS
# All options following a recipient name will be enabled for this recipient only.
# > recipient name=Name of this recipient.
# type=Choose between one of the scripts in the "senders" directory.
# destination directory=Directory where to put the backup in the other side.
# encrypt=Override main option for encryption. (true/false) (Optionnal option)
# specific option for type=See the chosen script to know what's the options.
# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)
# > recipient name=local example
# type=local
# destination directory=/my/local/backup
# encrypt=false
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
> recipient name=backup_server
type=rsync_ssh
destination directory=backup
# encrypt=false
ssh_host=192.168.0.4
ssh_user=backupuser
ssh_port=22
# ssh_key=/home/user/.ssh/id_rsa
ssh_pwd=iwefulhdskn!
# ssh_options=
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
# > recipient name=b2 example
# type=b2
# destination directory=backup
# encrypt=false
# b2_bucket=my-bucket
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=
any suggestions or tips? after changing this i've also rebooted...
Hi,
Is Archivist good solution when backing up file is near about 25Gb through ssh_rsync and there are frequent disconnects?
I am on the latest ynh install, stable. Getting this error everytime I update any app on ynh:
The YAML metadata file associated with logs is damaged: '/var/log/yunohost/categories/operation/20210220-233710-app_install-archivist.yml
Error: Corrupted YAML read from /var/log/yunohost/categories/operation/20210220-233710-app_install-archivist.yml (reason: expected '', but found ''
in "", line 1, column 13:
'**********'a'**********'r'**********'g'**** ...
^)'
Is this important? And is this the right place to report it? Cheers
Hellow
I found that the commit on community.json is outdated, hence people installing the app via this list won't benefit from the last upgrades / improvements ...
Please create a PR to update it on the list (I do not maintain this app so it's not relevant for me to do it and it might be refused...)
ProTip™ : possibly you can now set the commit to 'HEAD', to use the latest commit of master automatically in the future. If you do this, it is strongly recommended to use a 'testing' branch for changes that are not deemed stable yet.
I installed archivist some days ago and set the backup interval to daily. However, there are no backups from archivist to be found. I also uninstalled and reinstalled the app. Still, no backups. Should those backups be also visible in the web admin panel where the manually made Yunohost backups are visible?
I just installed the Archivist app. Uninstalled and reinstalled it again. Still, no backups.
Backups are being created.
There are no archivist logs being generated (looked in /var/log/archivist). The dir is empty.
Hi,
I add this at the beginning of archivist.sh :
echo() {
command echo
}
So the log show the time before each step and I can see how slow is my server when doing backup !!
If you mind, please add it to your script.
Cheers,
Cyril
This app needs some love to convert it to packaging V2 :)
This will require quite some work to convert it as it's non-standard, but most of it should be pretty straightforward.
Anyone interested to help ? :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.