Git Product home page Git Product logo

archivist_ynh's People

Contributors

alexaubin avatar ericgaspar avatar gofannon avatar kay0u avatar lapineige avatar maniackcrudelis avatar salamandar avatar yalh76 avatar yunohost-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

archivist_ynh's Issues

"Not enough free space on…" : add more storage space info

I would like to suggest an improvement to the log message we receive by email.

First of all, error messages such as "Not enough free space on…" could be highlighted (using bold ?) to catch them immediately when reading the email.

Secondly, this "Not enough free space on…" error message could show how big the archive was meant to be, and maybe how much space is available on the targeted storage.
That would help us diagnose this lack of space faster (and maybe detect any sudden drastic increase) to work on a workaround.

I guess it could be done by showing 1) Yunohost message about the final size of the backup 2) the result of df /storage/path command.
I know how to implement the second improvement, but not the first one : how do you catch that log ?

Question on operation with B2

Hi there, I'm wondering if there is some further documentation on using Archivist? It does not appear to be working, and I'm not sure whether I have done everything that I was supposed to do. I am assuming archivist will run when I generate a backup on YunoHost?

So far, I have taken the following steps:

  1. Install Archivist via YunoHost
  2. Configure Backup_list.conf (see below)
  3. Download B2 tool
  4. Created authorized user for B2
  5. Set up alias for B2 tool (b2="~/b2-linux")
  6. Run a YunoHost backup using web interface

Config:

## OPTIONS

# Backup directory.
backup_dir=/home/yunohost.app/archivist/backup

# Encrypted backup directory.
# Usually just next to the backup directory
enc_backup_dir=/home/yunohost.app/archivist/encrypted_backup

# Encryption
# true/false
encrypt=false
# Password for encryption
# This file should be set at chmod 400
cryptpass=na

# Compression algorithm to use
# gzip
#   Standard compression algorithm.
#   Fast and light on ressources.
# lzop
#   Very fast algorithm.
#   Low efficency
# zstd
#   Zstandard algorithm
#   Similar to gzip but faster
# bzip2
#   Very slow but very effective.
#   Light on ressources.
# lzma
#   Faster than bzip2 and even more effective.
#   But demanding in resources.
# lzip
#   Similar to lzma.
# xz
#   Similar to lzma but more recent.
#
# none
#   No compression
#   Concatenation in a tar file only
# symlink
#   No compression
#   Symbolic link to the YunoHost tar file from /home/yunohost.backup/archives
#   The symbolic link is used only for the local $backup_dir of archivist.
#   Real tar files are send to recipients.
#
#   This mode allow archivist to not duplicate locally the backup if not compressed.
#   WARNING: Does not work with encryption !
#
# Default: gzip
ynh_compression_mode=gzip
files_compression_mode=gzip

# --------------------------------------------------------------------------------------
## YUNOHOST BACKUPS

# YunoHost backups
# Specific backup made by 'yunohost backup' command
# ynh_core_backup=true/false: Make a backup of the core of YunoHost, without any apps.
ynh_core_backup=true
ynh_core_pre_backup=
ynh_core_post_backup=

# ynh_app_backup=APP: Make a backup of an app.
# You can add as many lines as you want to save multiple apps.
# List all apps with sudo yunohost app list --installed | grep "id: "
# ynh_app_backup=wordpress
# If you don't want an app with a lot of data to backup those big files, please add 'do_not_backup_data' after the app name.
# ynh_app_backup=transmission do_not_backup_data
ynh_app_pre_backup=
ynh_app_post_backup=
ynh_app_backup=alltube
ynh_app_backup=archivist
ynh_app_backup=cryptpad
ynh_app_backup=element
ynh_app_backup=encryptic
ynh_app_backup=gitea
ynh_app_backup=grocy
ynh_app_backup=lstu
ynh_app_backup=lufi
ynh_app_backup=piwigo
ynh_app_backup=rainloop
ynh_app_backup=restic
ynh_app_backup=searx
ynh_app_backup=strut
ynh_app_backup=synapse
ynh_app_backup=ttrss
ynh_app_backup=wallabag2
ynh_app_backup=zerobin
ynh_app_backup=

# --------------------------------------------------------------------------------------
## FILES BACKUPS

# Max size for each backup in Mb.
# Be carreful, it's only a soft limit, that means the script will try to limit each backup.
# Because, it will never made more than one backup for a single directory 
# (Even if it's a real big directory without subdirectories). 
# And the same if there's files next to subdirectories, only one backup will be made for all this files.
max_size=1000

# Backup of files and directories
# Add as many lines as you want for each directory or file you want to be backuped.
# Exclude a directory, a file, or multiples files with a regex with the exclude_backup instruction.
# file_to_backup="/directory/to backup"
# file_to_backup=/file/to backup
# exclude_backup="/directory/to exclude"
# exclude_backup=/file/to exclude
# exclude_backup="/exclude/all/tarball/in_this_directory/.tar.gz$"
files_pre_backup=
files_post_backup=
file_to_backup=
exclude_backup=

# --------------------------------------------------------------------------------------
## BACKUPS RECIPIENTS

# All options following a recipient name will be enabled for this recipient only.
# > recipient name=Name of this recipient.
# type=Choose between one of the scripts in the "senders" directory.
# destination directory=Directory where to put the backup in the other side.
# encrypt=Override main option for encryption. (true/false) (Optionnal option)
# specific option for type=See the chosen script to know what's the options.
# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)

# > recipient name=local example
# type=local
# destination directory=/my/local/backup
# encrypt=false
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

# > recipient name=rsync ssh example
# type=rsync_ssh
# destination directory=backup
# encrypt=false
# ssh_host=domain.tld
# ssh_user=user
# ssh_port=22
# ssh_key=/home/user/.ssh/id_rsa
# ssh_pwd=
# ssh_options=
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

 > recipient name=[B2 NAME]
 type=b2
 destination directory=/
 encrypt=false
 b2_bucket=[BUCKET NAME]
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

Archivist config panel not accessible

Describe the bug

YunoHost encountered an internal error when trying to access the config-panel pf archivist.

Context

  • Hardware: VPS bought online
  • YunoHost version: 4.3.6.2
  • I have access to my server: Through SSH + through the webadmin
  • Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: no
  • Using package version/branch: 1.3.2~ynh1

Steps to reproduce

  • When trying to access the config-panel in archivist I get the following error message:

Config panel question 'encrypt' should be initialized with a value during install or upgrade.

  • I did the following:
    1. Go to: mywebsite/yunohost/admin/#/apps/archivist
    2. Click on "config-panel"
    3. See this error:

archivist config panel

Backup error for app seafile

Hello everyone,
when I try to backup my apps, archivist has this error when creating a temporary backup for the app seafile.

Traceback (most recent call last):                                                                                             │# type=Choose between one of the scripts in the "senders" directory.
  File "/usr/bin/yunohost", line 72, in <module>                                                                               │# destination directory=Directory where to put the backup in the other side.
    parser=parser                                                                                                              │# encrypt=Override main option for encryption. (true/false) (Optionnal option)
  File "/usr/lib/moulinette/yunohost/__init__.py", line 29, in cli                                                             │# specific option for type=See the chosen script to know what's the options.
    top_parser=parser                                                                                                          │# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
  File "/usr/lib/python2.7/dist-packages/moulinette/__init__.py", line 120, in cli                                             │# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)
    args, output_as=output_as, timeout=timeout                                                                                 │
  File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/cli.py", line 477, in run                                       │> recipient name=Fritzbox NAS
    ret = self.actionsmap.process(args, timeout=timeout)                                                                       │type=local
  File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 592, in process                                       │destination directory=/mnt/fritzbox/TimeMachine/yh-backups
    return func(**arguments)                                                                                                   │# encrypt=false
  File "/usr/lib/moulinette/yunohost/backup.py", line 2062, in backup_create                                                   │# pre_backup=
    backup_manager.backup()                                                                                                    │# post_backup=
  File "/usr/lib/moulinette/yunohost/backup.py", line 730, in backup                                                           │# exclude backup=/exclude/dir
    method.mount_and_backup()                                                                                                  │# include backup=
  File "/usr/lib/moulinette/yunohost/backup.py", line 1574, in mount_and_backup                                                │
    self.backup()                                                                                                              │# > recipient name=local example
  File "/usr/lib/moulinette/yunohost/backup.py", line 1745, in backup                                                          │# type=local
    shutil.copytree(source, dest)                                                                                              │# destination directory=/my/local/backup
  File "/usr/lib/python2.7/shutil.py", line 231, in copytree                                                                   │# encrypt=false
    raise Error, errors                                                                                                        │# pre_backup=
shutil.Error: [('/opt/yunohost/seafile/ccnet/ccnet.sock', '/home/admin/temp/ynh_backup/temp/apps/seafile/backup/opt/yunohost/se│Backup_list.conf                                                                                             122,1          76%
afile/ccnet/ccnet.sock', "[Errno 6] No such device or address: '/opt/yunohost/seafile/ccnet/ccnet.sock'")]                     │
>>> The temporary backup failed..

In my Backup_list.conf I added a pre-backup action to stop both seafile services: seafile and seahub.

ynh_app_pre_backup=/etc/backups/services-stop.sh
ynh_app_post_backup=/etc/backups/services-start.sh
ynh_app_backup=archivist
ynh_app_backup=seafile do_not_backup_data

Backing up the seafile app via command line BACKUP_CORE_ONLY=1 yunohost backup create --app seafile works flawlessly.

Any ideas or suggestions what is going wrong?

Problem Description: The MongoDB service is shown as not found or inactive despite its proper functionality

Problem Description:
The MongoDB service is shown as not found or inactive despite its proper functionality. When loading the configuration panel, an error message appears stating: “Config panel question ‘encrypt’ should be initialized with a value during install or upgrade.”

mey system https://paste.yunohost.org/raw/xihaziyavu
$ systemctl status mongod
● mongod.service
Loaded: not-found (Reason: Unit mongod.service not found.)
Active: active (running) since Tue 2023-05-16 10:37:20 UTC; 5 days ago
Main PID: 1135325
CPU: 58min 36.292s
CGroup: /system.slice/mongod.service
└─1135325 /usr/bin/mongod --config /etc/mongod.conf
carrabelloy@srv:
$

https://forum.yunohost.org/t/unkown-status-for-mongodb/24837?u=carrabelloy

How to post a meaningful bug report

  1. Read this whole template first.
  2. Determine if you are on the right place:
    • If you were performing an action on the app from the webadmin or the CLI (install, update, backup, restore, change_url...), you are on the right place!
    • Otherwise, the issue may be due to the app itself. Refer to its documentation or repository for help.
    • When in doubt, post here and we will figure it out together.
  3. Delete the italic comments as you write over them below, and remove this guide.

Describe the bug

A clear and concise description of what the bug is.

Context

  • Hardware: VPS bought online / Old laptop or computer / Raspberry Pi at home / Internet Cube with VPN / Other ARM board / ...
  • YunoHost version: x.x.x
  • I have access to my server: Through SSH | through the webadmin | direct access via keyboard / screen | ...
  • Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: no / yes
    • If yes, please explain:
  • Using, or trying to install package version/branch:
  • If upgrading, current package version: can be found in the admin, or with yunohost app info $app_id

Steps to reproduce

  • If you performed a command from the CLI, the command itself is enough. For example:
    sudo yunohost app install the_app
  • If you used the webadmin, please perform the equivalent command from the CLI first.
  • If the error occurs in your browser, explain what you did:
    1. Go to '...'
    2. Click on '...'
    3. Scroll down to '...'
    4. See error

Expected behavior

A clear and concise description of what you expected to happen. You can remove this section if the command above is enough to understand your intent.

Logs

When an operation fails, YunoHost provides a simple way to share the logs.

  • In the webadmin, the error message contains a link to the relevant log page. On that page, you will be able to 'Share with Yunopaste'. If you missed it, the logs of previous operations are also available under Tools > Logs.
  • In command line, the command to share the logs is displayed at the end of the operation and looks like yunohost log display [log name] --share. If you missed it, you can find the log ID of a previous operation using yunohost log list.

After sharing the log, please copypaste directly the link provided by YunoHost (to help readability, no need to copypaste the entire content of the log here, just the link is enough...)

If applicable and useful, add screenshots to help explain your problem.

How to restore

It may be a silly question, but how to restore these backups?

Integration in Yunohost auto-diagnosis ? Warn if new app has no backup

Hello,

I am wondering if an app can register something for the auto-diagnosis screen (@YunoHost-Apps/apps-group ?).
If so, could it be that this app would add a check in that diagnosis that would scan installed apps, and warn if automated backup is not set up for one of them.
This way, if you install a new app after archivist first config, you don't have to think about adding it to automated backups : the diagnosis will warn you have this app having no backup setup.

What do you think about such a feature ?

Do not create real backup if temp failed

Hi,

I launched /opt/yunohost/archivist/archivist.sh after removing my Laverna app, and without editing the conf file (yeah, I should have, I simply forgot). I already made a backup with archivist, some weeks ago.

Here is the output:

>> Make a temporary backup for laverna_backup
Error: App 'laverna' will not be saved
Error: There is nothing to save
rm: cannot remove ‘/home/yunohost.app/archivist/backup/ynh_backup/temp/info.json’: No such file or directory
find: `/home/yunohost.app/archivist/backup/ynh_backup/temp': No such file or directory
>>> This backup is different than the previous one
>>>> Make a real backup for laverna_backup
Error: App 'laverna' will not be saved
Error: There is nothing to save
cp: cannot stat ‘/home/yunohost.backup/archives/laverna_backup.tar.gz’: No such file or directory
cp: cannot stat ‘/home/yunohost.backup/archives/laverna_backup.info.json’: No such file or directory
1.7M /home/yunohost.app/archivist/backup/ynh_backup/laverna_backup.tar.gz

If "Make a temporary backup" fails, maybe we should not run "Make a real backup" ?

Error in log

Hello,
I have these messages in my logs :

Erreur : Script de sauvegarde ‘conf_ssh’ inconnu
Erreur : Script de sauvegarde ‘conf_ssowat’ inconnu
Erreur : Script de sauvegarde ‘conf_ynh_firewall’ inconnu
Erreur : Script de sauvegarde ‘conf_xmpp’ inconnu
Erreur : Script de sauvegarde ‘conf_nginx’ inconnu
Erreur : Script de sauvegarde ‘conf_cron’ inconnu
Erreur : Script de sauvegarde ‘conf_ynh_currenthost’ inconnu

Not sure if this is an issue for the core backup.

6ril

../apps/archivist/config-panel

When I try to open the experimental config panel on apps/archivist/config-panel, the server does not load nothing, keeps on loading.

"host": "Debian 9.11",
"kernel": "4.9.0-11-amd64",
"packages": {
"yunohost": {
"repo": "stable",
"version": "3.6.4.6"
},
"yunohost-admin": {
"repo": "stable",
"version": "3.6.4"
},
"moulinette": {
"repo": "stable",
"version": "3.6.4.1"
},
"ssowat": {
"repo": "stable",
"version": "3.6.4"

Feature request : add compression

Problem: As Yunohost 4.1 is out now, backups are no longer compressed https://forum.yunohost.org/t/yunohost-4-1-release-sortie-de-yunohost-4-1/13893
We can still compress them afterwards, but it adds some (manual) extra work.

For regular automated backup it can lead to an important storage space use, or would require manual work to fix this (then using this app would no longer be as "install&forget" than it was before).

Proposal : Would it be possible to add an option to compress backups ?
I'm willing to help testing it :)

Benefits : this would save a lot of storage space, while not requiring users to regularly think about it and spare some time doing it manually (with CLI).

Additional proposal : use Zstandard compression instead of Gzip to reduce CPU usage and storage space (detailed proposal here https://forum.yunohost.org/t/backups-implementing-zstandard-compression-instead-of-gzip/13379)

Rsync SSH Backup doesn’t work for some reason.

Not sure why it doesn't transfer backup to SSH server via rsync.. here's my config from /opt/yunohost/archivist/Backup_list.conf

## OPTIONS

# Backup directory.
backup_dir=/home/yunohost.app/archivist/backup

# Encrypted backup directory.
# Usually just next to the backup directory
enc_backup_dir=/home/yunohost.app/archivist/encrypted_backup

# Encryption
# true/false
encrypt=true
# Password for encryption
# This file should be set at chmod 400
cryptpass=/opt/yunohost/archivist/passkey

# --------------------------------------------------------------------------------------
## YUNOHOST BACKUPS

# YunoHost backups
# Specific backup made by 'yunohost backup' command
# ynh_core_backup=true/false: Make a backup of the core of YunoHost, without any apps.
ynh_core_backup=true
ynh_core_pre_backup=
ynh_core_post_backup=

# ynh_app_backup=APP: Make a backup of an app.
# You can add as many lines as you want to save multiple apps.
# List all apps with sudo yunohost app list --installed | grep "id: "
# ynh_app_backup=wordpress
# If you don't want an app with a lot of data to backup those big files, please add 'do_not_backup_data' after the app name.
# ynh_app_backup=transmission do_not_backup_data
ynh_app_pre_backup=
ynh_app_post_backup=
ynh_app_backup=archivist
ynh_app_backup=portainer
ynh_app_backup=

# --------------------------------------------------------------------------------------
## FILES BACKUPS

# Max size for each backup in Mb.
# Be carreful, it's only a soft limit, that means the script will try to limit each backup.
# Because, it will never made more than one backup for a single directory (Even if it's a real big directory without subdirectories). And the same if there's files next to subdirectories, only one backup will be made for all this files.
max_size=500

# Backup of files and directories
# Add as many lines as you want for each directory or file you want to be backuped.
# Exclude a directory, a file, or multiples files with a regex with the exclude_backup instruction.
# file_to_backup="/directory/to backup"
# file_to_backup=/file/to backup
# exclude_backup="/directory/to exclude"
# exclude_backup=/file/to exclude
# exclude_backup="/exclude/all/tarball/in_this_directory/.tar.gz$"
files_pre_backup=
files_post_backup=
file_to_backup=
exclude_backup=

# --------------------------------------------------------------------------------------
## BACKUPS RECIPIENTS

# All options following a recipient name will be enabled for this recipient only.
# > recipient name=Name of this recipient.
# type=Choose between one of the scripts in the "senders" directory.
# destination directory=Directory where to put the backup in the other side.
# encrypt=Override main option for encryption. (true/false) (Optionnal option)
# specific option for type=See the chosen script to know what's the options.
# exclude backup=Exclude a backup file or a directory from the backup. This option can be duplicated. (Optionnal option)
# include backup=Choose the only file or directory which be send. This option can be duplicated. (Optionnal option)

# > recipient name=local example
# type=local
# destination directory=/my/local/backup
# encrypt=false
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

 > recipient name=backup_server
 type=rsync_ssh
 destination directory=backup
# encrypt=false
 ssh_host=192.168.0.4
 ssh_user=backupuser
 ssh_port=22
# ssh_key=/home/user/.ssh/id_rsa
 ssh_pwd=iwefulhdskn!
# ssh_options=
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

# > recipient name=b2 example
# type=b2
# destination directory=backup
# encrypt=false
# b2_bucket=my-bucket
# pre_backup=
# post_backup=
# exclude backup=/exclude/dir
# include backup=

any suggestions or tips? after changing this i've also rebooted...

Getting this error recently

I am on the latest ynh install, stable. Getting this error everytime I update any app on ynh:

The YAML metadata file associated with logs is damaged: '/var/log/yunohost/categories/operation/20210220-233710-app_install-archivist.yml
Error: Corrupted YAML read from /var/log/yunohost/categories/operation/20210220-233710-app_install-archivist.yml (reason: expected '', but found ''
in "", line 1, column 13:
'**********'a'**********'r'**********'g'**** ...
^)'

Is this important? And is this the right place to report it? Cheers

Commit is not up to date in the community.json

Hellow

I found that the commit on community.json is outdated, hence people installing the app via this list won't benefit from the last upgrades / improvements ...

Please create a PR to update it on the list (I do not maintain this app so it's not relevant for me to do it and it might be refused...)

ProTip™ : possibly you can now set the commit to 'HEAD', to use the latest commit of master automatically in the future. If you do this, it is strongly recommended to use a 'testing' branch for changes that are not deemed stable yet.

Archivist created no backups

Describe the bug

I installed archivist some days ago and set the backup interval to daily. However, there are no backups from archivist to be found. I also uninstalled and reinstalled the app. Still, no backups. Should those backups be also visible in the web admin panel where the manually made Yunohost backups are visible?

Context

  • Hardware: *VPS bought online
  • YunoHost version: 1.11.20
  • I have access to my server: *Through SSH | through the webadmin
  • Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: *no
  • Using, or trying to install package version/branch: 1.3.3~ynh1

Steps to reproduce

I just installed the Archivist app. Uninstalled and reinstalled it again. Still, no backups.

Expected behavior

Backups are being created.

Logs

There are no archivist logs being generated (looked in /var/log/archivist). The dir is empty.

Not a Issue but I wasn't able to create a PR ....

Hi,
I add this at the beginning of archivist.sh :
echo() {
command echo $(date +%T) ": $@"
}
So the log show the time before each step and I can see how slow is my server when doing backup !!

If you mind, please add it to your script.

Cheers,
Cyril

Update to packaging V2

This app needs some love to convert it to packaging V2 :)

This will require quite some work to convert it as it's non-standard, but most of it should be pretty straightforward.

Anyone interested to help ? :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.