google / recursive-version-control-system Goto Github PK

The export and import commands allow you to write/read a subset of the versioned files to/from a zip file.

That, in turn, is used by the push and pull helper for local files.

We should integrate with the age encryption tool to allow those zip files to be encrypted, and correspondingly update the rvcs-push-file and rvcs-pull-file extensions to support this new encryption feature.

We should create sign/verify helpers that integrate with Sigstore.

At a minimum I would like to have some support for keyless signing similar to what https://github.com/wlynch/smimecosign provides for git.

Additionally, it might be good to have any other sign/verify helpers (such as the ones for SSH keys) write entries to the Rekor transparency log that can be later verified.

Expected Behavior

The user's ~/.rvcs/archive directory only maintains the history that the user still cares about, and does not require an excessive amount of additional storage versus the user's actual file contents.

Actual Behavior

The user's ~/.rvcs/archive directory grows without bounds and contains full copies of every version of every file tracked.

Steps to Reproduce the Problem

To reproduce the issue of the directory growing without bound:

1. Create a temporary directory
2. Create a file within that directory
3. Snapshot that file (but not the containing directory)
4. Delete that file
5. Snapshot the (now empty) temporary directory
6. The snapshot of the now-deleted file will not be reachable from the snapshot of the directory, but it will also never be garbage collected.

To reproduce the issue of a full copy of every version of every file being retained:

1. Create a temporary file
2. Add some text to it
3. Snapshot the file
4. Add a single character to the end of the file
5. Snapshot the file again
6. There will be two separate entries under ~/.rvcs/archive/objects/ for the contents of the two different versions of the file which differ by only one character.

One way that we could fix both issues would be to add something like a rvcs cleanup command that would do the following:

1. Traverse all of the paths under ~/.rvcs/archive/paths/, and find all of the objects reachable from those paths
2. Add all of those reachable objects to a zip file stored under the ~/.rvcs/archive/ directory
3. Remove the now-zipped objects from the ~/.rvcs/archive/objects/ directory
4. Remove any objects from ~/.rvcs/archive/objects/ that were not reachable from any of the paths
5. Similarly remove any entries from ~/.rvcs/archive/paths/ and ~/.rvcs/archive/cache that do not correspond to any currently mapped paths

For the last two steps, we would need to take extra care to make sure that we do not have any race conditions with simultaneous runs of the rvcs snapshot or rvcs merge commands.

Additionally, we would need to extend the tool to be able to read objects from the zip file under the ~/.rvcs/archive/ directory if they are not in the ~/.rvcs/archive/objects/ directory.

Hello!
This is very interesting project. Can you create some easy issues and labels like 'good first issue' etc.?
I will be glad to contribute.

The rvcs local file storage uses two separate subdirectories for storing data:

1. objects for objects less than 1MB in size.
2. largeObjects for all objects larger than 1MB.

This is so that the largeObjects subdirectory can be made into a symlink pointing to an external storage service (such as an external drive or network-attached-storage), and prevent the rvcs archive from taking up all of the space on your local disk.

However, you probably want files to be encrypted before being written to an external storage system, so we should support that.

We should add some sort of support for configuration settings.

Things that I can think of that would make for good options to have settings for:

1. Paths to exclude from snapshotting. For example, you probably don't want to snapshot your ~/.Trash directory
2. A block list of objects to always exclude from bundle exporting/importing
3. A default hash function (when we eventually support more than just sha256)

0b6fc6c