JotFS is a deduplicating file storage engine backed by S3. It works by splitting incoming files into small content-defined chunks and only storing those chunks which it has not seen yet.

For a short explanation of how JotFS works, please see the introductory article.

Features:

• Reduces storage space
• Reduces upload bandwidth
• Backed by S3 or S3 compatible storage (Minio, GCP etc.)
• Client library available for Go (more languages planned)
• Client CLI with familiar commands: jot cp, jot ls etc.
• Optional file versioning
• Easy deployment with a single binary or Docker image

JotFS is currently beta level software.

docker pull jotfs/jotfs

Download the latest binary from the releases page and extract. Example:

gzip -dc ./jotfs_linux_amd64_v0.0.3.gz > jotfs
chmod +x ./jotfs
./jotfs -version

git clone https://github.com/jotfs/jotfs.git
cd jotfs
CGO_ENABLED=1 go build ./cmd/jotfs

Using AWS S3 backed storage (see the wiki for more examples and advanced configuration):

jotfs -store_bucket="jotfs-test"

Use the jot CLI to interact with the server:

jot cp data.txt jot://data.txt

jot ls /

jot cp jot://data.txt data_download.txt

The server stores metadata in a database file located at ./jotfs.db by default. When running the Docker image, you should mount a volume to /app in the container so the database is persisted between runs:

docker run -v jotfs:/app jotfs/jotfs <FLAGS...>

Contributions to JotFS and its client applications are welcome. Please open an issue if you would like to report bugs or suggest new features.

JotFS is licensed under the Apache 2.0 License. See LICENSE for details.