This program is meant to backup smaller online storage to larger offline storage. I use it to backup a 5TB drive to an offline 8TB disk. It uses the remaining 3TB to store older files that have been deleted from the 5TB drive.

Files are copied to the backup and optionally verified. Files that exist on offline storage but don't exist online are preserved until the offline disk becomes full. When the offline disk becomes full, the oldest files are deleted to make room for newer files.

This allows me to store the most recent 5TB of data will storing the next older 3TB offline without me having to manage it manually.

If you have a modern linux system, you should be able to just type make. It has no special dependencies. It shouldn't be hard to build for other unix systems.

It doesn't use mmap (mainly so errors are easier to detect) so should be relatively easy for non-unix systems as well.

make

To run this for the first time, try the following commands. This won't write anything to disk.

./offlinebackup --help
./offlinebackup --srcdest . /tmp --dryrun

Usage: offlinebackup (OPTIONS)

Copy files from directory A to B.

If you provide multiple --srcdest arguments, then PLEASE make sure no A's or B's overlap in the filesystem. Also ensure that all B directories are on the same partition so free space calculations make sense.

Prompt user before deleting or copying

disable --interactive (default)

Print progress to stdout

disable --verbose (default)

Print memory stats to stdout

disable --memstats (default)

Re-read files after writing them to verify contents

disable --verifycopy (default)

Compare existing destination files to source files

disable --verifymatch (default)

Quit out if existing files don't verify, requires/enables --verifymatch (default)

disable --mismatchquit

Go through the motions but don't delete or copy

disable --dryrun (default)

Copy files from source to destination (default)

disable --copyfiles

Delete destination files that don't match source (default)

disable --deletemistmatch

Delete oldest destination files to make space on device (default)

disable -deletetofree

Allow 0-byte files on source to match destination files (default)

disable --allow0match

Truncate source files older than 183 days to 0 bytes

All --maxXXX options also enable --allow0match --verifymatch

Truncate source files older than 366 days to 0 bytes

All --maxXXX options also enable --allow0match --verifymatch

Truncate source files older than 548 days to 0 bytes

All --maxXXX options also enable --allow0match --verifymatch

Truncate source files older than 731 days to 0 bytes

All --maxXXX options also enable --allow0match --verifymatch

Do not truncate source files (default)

Attempt copying even if it looks like we'll run out of space

Shortcut for --not-copyfiles --verbose --verifymatch --not-deletemismatch --allow0match

Shortcut for --verbose --verifycopy --verifymatch --allow0match --max6months