Right now, there is a single command gitfs. This starts a file system server on a designated mountpoint. Ideally, we want to support a variety of commands, not just starting the server. For example:

• start would perform the same functionality as gitfs does today. If no command is provided, then this command should be invoked by default.
• stop would force the file system server to unmount. Note, that using fuse.Unmount will not work as it requires all references to the underlying files to be released (i.e all terminal windows and applications closed). I've been able to do this on osx and linux using the sudo umount -f ${mountpoint}
• config would behave a lot like git-config, making it possible to modify properties without needing to modify the files directly.

Each of these commands should be implemented as their own ticket.

You can find a copy of many github repositories on bigquery using the open github dataset. Let's add a remote that can support querying bigquery for these repositories.

[jpitz@jpitz ~/Development/code/github.com/mjpitz/mjpitz.com master 0/0]
$ git add _config.yml
fatal: error when closing sha1 file: Input/output error

Nothing coming through on the error logs so definitely need to add some better logging to help track this down.

The configuration should already be defined and there is a placeholder in the pkg/remotes/remotes.go file for adding the construction of the bitbucket client. I was basing the config off of go-bitbucket.

Right now, we clone on the fly. Some people would like to have some repos cloned automatically. Seems like adding support for this would be easy. Under the clone config, we can add an init section that has a list of strings / regex. If the repo url matches the string or regex, then the repo is added to a list to be cloned once the fs starts up.

Prereq: #13 as it will involve the introduction of the framework.

Implement the config command as described in #12

Right now, we rely on your ssh keys to be added to the session. When gitfs starts up, the terminal won't have the ssh keys to work with and and so all clones will fail.

This should let us use travis to produce binaries for the proper platforms. We can use gox to do cross compilation and we can target the appropriate platforms.

When I'm on a "user" directory, an ls causes each repo to be cloned.

During gh-19, I had refactored the cloning logic and left room for the addition of svn and hg repo management. In this issue, let's implement the svncloner. I'd start by looking at the existing gitcloner and compositecloner implementations.

I believe that this is because we don't have a proper inode system in place.

When I run something like jekyll serve, files aren't automatically updating. This is likely because the modification times aren't being updated accordingly.

Right now, we only support leveraging the memfs implementation for the go-git integration. In this change we should handle a couple things.

1. establish configuration for swapping out the billy filesystem layer in go-git
2. making the storage layer construction pluggable

Right now we only support shallow clones. Lets work through the requirements for adding support for non-shallow clones and the configuration that we would like to override on a per repository basis. I.e some repos you may never want to full clone. Default should be to shallow clone to keep the fs responsive.

This has fallen a fair bit behind compared to some of the older projects. We will need to go through and update this project to use some of the latest libraries. There will be a fair number of configuration changes that need to be made.

Implement the start command as described in #12

Currently pointed at mjpitz

I started a docs directory that is currently used to power https://mjpitz.github.io/gitfs. In this ticket, let's get a project site up and running.

Was trying to go through a few projects gh-pages. After doing a sed operation, the permissions of a file appear to be stripped:

$ ls -la _layouts/
total 32
drwxr-xr-x   4 jpitz  staff   128 May  2 06:46 .
drwxr-xr-x  13 jpitz  staff   416 May  2 06:45 ..
----------   1 jpitz  staff  5177 May  2 06:46 default.html
-rw-r--r--   1 jpitz  staff  4228 May  2 06:48 reference.html

Not sure what happened there but should be easy to reproduce:

$ ag -l <pattern> | xargs sed -i "s:<pattern>:<replace>:g"

gox won't be able to compile it, but we can do it using go build directly.

I noticed that after making a change, there was a lock lingering around. Seems like something is off.

[mjpitz@mjpitz ~/Development/code/github.com/mjpitz/mjpitz.com master 1/1]
$ git add .
fatal: Unable to create '/Users/mjpitz/Development/code/github.com/mjpitz/mjpitz.com/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

Right now, we always checkout at HEAD. It would be nice to be able to maintain a notion of a "stable" position for each repo.

We will need a pretty large design effort to get this kind of feature built in.

This may be two or one issues - split it if needed.

$ cat ~/.gitfs/config.yml:
mount: "${HOME}/GitFS"
accounts:
  - github:
      oauth2:
        token: ...
      users:
      - peter-kehl 
clone:
  repositoryRoot: "${HOME}/.gitfs/cache" # this is where your repos will be cloned
  depth: 0 # depth 0 means full clone
  
$ ~/Downloads/gitfs_linux_amd64 start
INFO[0000] [main] configured mount point: /home/pkehl/GitFS 
INFO[0000] [main] fetching repositories
...
NFO[0005] [main] now serving file system               
ERRO[0026] [filesystem.directory] failed to clone url: [email protected]:peter-kehl/docker.github.io.git, failed to clone repo: [email protected]:peter-kehl/docker.github.io.git: ssh: handshake failed: knownhosts: key is unknown

That did create a directory structure under ~/GitFS, with my repositories - both my own and in my organisations. However, all are empty:

$ ls /home/pkehl/GitFS/github.com/peter-kehl/4clojure
# -> empty output
$ grep 4clojure ~/.gitfs/cache/*/.git/config
N19UKEAUM8PTKEEIH4CRI8Q5HE9FIHVPUN99BR49MQPJ7DE6ME6G/.git/config:	url = [email protected]:peter-kehl/4clojure.git
$ ls -a ~/.gitfs/cache/N19UKEAUM8PTKEEIH4CRI8Q5HE9FIHVPUN99BR49MQPJ7DE6ME6G
.  ..  .git

$ ~/Downloads/gitfs_linux_amd64 stop
INFO[0000] Unmounting /home/pkehl/GitFS

That shows the same even when repeated.

However, it didn't kill the server process. Even worse, after I Ctrl+C stopped the main process, I couldn't restart it:

$ ~/Downloads/gitfs_linux_amd64 start
INFO[0000] [main] configured mount point: /home/pkehl/GitFS 
INFO[0000] [main] fetching repositories                 
INFO[0000] [remotes.github] processing organizations for user: peter-kehl 
INFO[0000] [remotes.github] processing repositories for user: peter-kehl 
INFO[0001] [remotes.github] processing repositories for organization: Melati 
INFO[0001] [remotes.github] processing repositories for organization: SeLite 
INFO[0002] [remotes.github] processing repositories for organization: VanClojure 
INFO[0003] [remotes.github] processing repositories for organization: DaisyDiff 
INFO[0003] [remotes.github] processing repositories for organization: PanEris 
INFO[0003] [main] parsing repositories into a directory structure 
INFO[0003] [main] attempting to mount /home/pkehl/GitFS 
2019/01/28 09:11:40 mount helper error: fusermount: failed to access mountpoint /home/pkehl/GitFS: Permission denied
ERRO[0003] [main] failed to mount mountpoint: fusermount: exit status 1

Why is that worth fixing? A user is likely to start gitfs right after rebooting. On laptops, some Linux distributions (Manjaro/Arch) start with disabled Wifi. That means the user user has to reboot again...

This may be an issue with the underlying library, but I made a whole change on a repo and then went to push it and ran into this issue. At first I thought it was the integration with gitfs, but when I looked at the underlying directory, it has the same issue.

[mjpitz@mjpitz ~/.gitfs/cache/DA0KGNSRU2DH2QGP0N4INC1225L3ALUVUJNED5PK7JTJTJ8I00B0 master 1/1]
$ git push
fatal: No configured push destination.                                                                                                                                                              
Either specify the URL from the command-line or configure a remote repository using

    git remote add <name> <url>

and then push using the remote name

    git push <name>

Many projects have a single install.sh script that can be curl'd and piped into bash to install their application. We should add one to make it easy for folks to install without needing to install go.

Also update documentation accordingly.

Right now, we don't tell people how to contribute to the project and we should.

During gh-19, I had refactored the cloning logic and left room for the addition of svn and hg repo management. In this issue, let's implement the hgcloner. I'd start by looking at the existing gitcloner and compositecloner implementations.

Right now, we don't communicate how to gracefully stop the server. We should add a section to the readme.

The configuration should already be defined and there is a placeholder in the pkg/remotes/remotes.go file for adding the construction of the gitlab client. I was basing the config off of go-gitlab.

Right now, we require third party dependencies to be installed on the developer machine. Let's encapsulate those into a docker build image so that engineers can develop without needing to install dev-specific tooling.

Right now, we don't use a linter and we really should. I know there is a good number of things that will fail the linters check. Todo:

• add linter to make target
• swap travis build over to using the make targets
• fix issues identified by linter

Right now we only support parsing ssh urls. We should add support to parse https urls.

Dear Mya,

Thank you for GitFS.

Please, could you document scope(s) of GitHub token. It seems that "read:user" scope is enough.

Should probably do #20 before completing this.

Set up a timed interval to re-pull remotes and add new projects where necessary. We should never delete projects form the local system as they may contain changes and we want to give engineers the opportunity to update their references where necessary.

Right now, when we discover and parse a url, each node in the directory gets assigned a new inode id with each iteration. While this isn't a problem for our current usage, it will pose a bigger issue if we ever want to update the directory tree on the fly.

There's not much there right now: https://mjpitz.github.io/gitfs/

It's currently being generated off of the /docs directory.

Prereq: #13 as it will involve the introduction of the framework.

Implement the stop command as described in #12