nbari / backup Goto Github PK

define db (sqlite or similar) to keep track of keys and paths where backups are stored (probably multiple destinations)

also encrypt the db since files can only be decrypted with it

test encrypting in chunks, benchmark vs AES

keep track of changes and update/create backup, do this async:

• keep track of changes in DB
• digest DB data at custom time (define priority)

define the backup layout:

• collect files and make it possible to restore to a defined root
• store the backups in a distributed tree (optimize storage) (check https://github.com/awslabs/mountpoint-s3)

zstd offers a good ratio(speed, size), maybe also use xv to get a better compression ratio, (find way to provide multiple option)

flow:

• get files
• compress
• encrypt

if the system to backup is almost full, give the option to select a buffer dir in which compression and encryption occur.

The flow:

1. get checksum
2. compress (split if necessary (define maybe max chunk/spit size)
3. upload