One proposal

What if we referred to every type of object the same way we reference files, using the URL. The general idea is that the types are referred to hierarchically. I think this maps fairly well to the concepts we have. For example to access the filesystem at a specific commit we use a file subroute of commit:

# access <file> as it was in <commit>
curl pfs/commit/<commit>/file/<file>
# the default value for <commit> is still `"master"`:
curl pfs/file/<file> # reads from master
# so that use case doesn't get longer

Here's what it would look like:

• /file

# Access to master doesn't change
curl -XPUT pfs/file/<file> -d @<local_file>
curl -XGET pfs/file/<file>
# Accessing outdated data does
curl -XGET pfs/commit/<commit>/file/<file>
# And writing to other branches does as well
curl -XGET pfs/branch/<branch>/file/<file>

• /commit

# committing a branch is now
curl -XPOST pfs/branch/foo/commit
# committing master remains unchanged
curl -XPOST pfs/commit
# Listing commits remains unchanged
curl -XGET pfs/commit
# Getting commits from <branch>
curl -XGET pfs/branch/<branch>/commit

• /branch

# creating <branch> from <commit>
curl -XPOST pfs/commit/<commit>/branch
# writing <file> to <branch>
curl -XPUT pfs/branch/<branch>/file/<file> -d @<local_file>

from pachyderm.

This has a very natural extension to the next primitive we're going to add which is jobs. Here's how the /job route will look in this scheme I think:

# scheduling <job> on <master>
curl -XPUT pfs/job/<job> -d @<local_job>
# scheduling <job> on <branch>
curl -XPUT pfs/branch/<branch>/job/<job> -d @<local_job>
# accessing <file> from the output of <job>
curl -XGET pfs/job/<job>/file/<file>
# this filesystem is read-only
curl -XPUT pfs/job/<job>/file/<file> -d @<local_file>
Method not allowed.

from pachyderm.

Looks interesting. It would be great if compatibility is kept between git's http access syntax and the proposed file access syntax. For example, git's way of accessing the ReadMe.md at master and some old version is

    https://github.com/pachyderm/pfs/blob/master/README.md
    https://github.com/pachyderm/pfs/blob/90fcb0f0d8ab233daf2449c1125c5e6f7bee0d2d/README.md

Essentially it is following the <root>/<branch>/<filename> pattern.

Since the syntax we select for the access API is going to effect the performance and other factors it may be better to evaluate all the options w.r.to some of the below factors.

Few important factors to consider for evaluation are:

• subscribers or listeners for any modifications at any level of the file system tree.
• security / access control
• Throttling / Rate limiting
• Caching / invalidating

Please add more if I missed something.

On the other hand, Most of these above are common for all file-systems since old ages - what is the need of hour and expected desperately from new file systems is:

• ability to join and remove additional file systems on the fly (e.g. files mapped on smart phones' SD card being able to join and remove as user walks across wifi zones). The key is: being able to serve files from the near-by smart phone's SD card on wifi, rather than going through central server from remote location (similar to bit-torrent), and fall back to central server when the phone is not available.

It would be great if new systems such as pfs can at least take one step towards that direction, if not full solve them.

from pachyderm.