This is in the old PR for the first UI prototype. It would still be a good thing to have for the new simple UI, to catch the most basic class of syntax errors before attempting to mount a new instance.

Would be nice if this was available as a part of core-list, but we may have to implement our own solution. Requires some tricky logic, as it needs to be restored if the user deletes all buckets.

See googlearchive/core-list#12

See: http://updates.html5rocks.com/2014/08/Easier-ArrayBuffer---String-conversion-with-the-Encoding-API

They're registered for unit tests but not with the actual FSP

When the FSP API is updated to support write operations, both providers should be updated to reflect this.

Operations to implement:

Basic

• Create
• Write
• Truncate

Non-recursive

• Delete
• Copy
• Move: implemented in #93

Recursive

These work by default for WebDAV are difficult for S3 because there's no concept of a directory. Need to have a discussion on how to best implement.

• Delete
• Copy
• Move: trivial once COPY is implemented

In the file webdavfs/js/wdfs.js, the JSDocs for the functions only have the onSuccess and onError callbacks documented, not the path to the file or directory to be read.

Like #56, this can be switched to using the new TextEncoder API

Local should use Chrome and Firefox and rerun every time the files change.

CI should use Firefox only and run once then quit.

Eventually the UI will need to be added to the WebDAV provider too. However, since a lot of it will be copied over from the S3 provider, so we should make sure things are working well there to avoid duplicated work later.

Now that there's a UI, we should start thinking about internationalising it. There's already a few hardcoded strings, and the longer we leave the conversion process the more work it will be to switch over to a dynamic solution.

I'm guessing we should use chrome.i18n for this.

Hopefully Chrome can be used on Travis and we can run the integration tests there too

This is more open-source friendly as it doesn't require access to the testing bucket on my personal AWS account.

Methods calls that we'd need to implement are

• AWS.config.update method
• AWS.config.regionproperty
• AWS.S3 constructor
  • listObjects method
  • getObject method
  • headBucket method

Current system is slightly buggy, and loses track if the same item is selected twice. core-list's default implementation doesn't seem to work.

See googlearchive/core-list#13

It would be good to run the tests on a CI server like Travis.

Since it's open source, would this be okay @jmuk?

Currently it downloads the file and checks the length, we should just make a HEAD request to get the length as a number

Both existing providers need an icon to be displayed in the Chrome extensions list, app drawer etc.
My mockup for the S3 one is this:

Which I think is a good direction to go in. The current Google Drive icon in the Files app is the same: The flat grey folder icon with the drive logo in negative space. This one just swaps it out for the S3 logo. However it currently isn't sized correctly (I think icons for Chrome extensions need some padding?). It would be good to get a professional designer to do this. We also might need to get permission to use the S3 logo.

As for WebDAV, it's trickier. The official logo is really ugly:

So it might be better to just have the folder with the word 'DAV' in Roboto in negative space or something similar.

It will take a while for the design behind #13 to be formalised, and longer for it to be implemented. In the meantime, we should offer a very basic authentication system so people can at least try out an early version of the providers.

The idea is this:

1. Only support one instance per provider
2. Open a window prompting the user to enter credentials when the extension is installed
3. If the user enters invalid credentials, or any other error occurs, the window will reappear with an appropriate error message, until the user enters correct credentials or closes the window
4. If the user closes this window, the extension goes into a deactivated state
5. The filesystem instance persists as long as the extension is installed, and remounts itself after any Chrome relaunches
6. To unmount the filesystem instance, the user uninstalls the extension
7. If the credentials ever expire or change, the extension will go into a deactivated state
8. When in the deactivated state caused by 4 or 7, the extension will prompt the user to enter new credentials as in step 2 whenever an attempt is made to use it

Currently displays error messages manually with paper-toast as a workaround. Blocked on googlearchive/paper-input#58

When I try to make a request to the WebDAV test server to copy a file, I get a 500 (internal server error) error code back.

Here's the request:

verb: 'COPY'
url: 'http://localhost:8000/source.txt'
headers: {
   'Destination': 'http://localhost:8000/target.txt'
}

This seems to match the specification for the COPY request. I also can't work out how to enable logging in JSDAV, so it's hard to work out exactly what the internal server error is. I guess asking on the JSDAV issues list about how to enable logging would be a good first step for this.

The documentation doesn't yet cover the expected behaviour of the write operations, only the options object that they will receive. Here are the questions I've come up with while working on them

cc @mtomasz-chromium

1. When copying a file or directory to a path that already exists, is it okay to overwrite what's already there? Do we need a flag for this?
2. How is Abort supposed to work? As far as I know, none of the APIs we're working with allow in-progress operations to be cancelled. Should Abort allow it to finish and then perform another operation to rollback the change in this case?
3. Same as 1. but for Move.
4. When opening a file with create: false for reading or writing, should a check be made at the time to ensure the file exists? Or should it wait until an attempt is made to access (read/write) the file before the error is returned?
5. What does exclusive mean in createDirectory?
6. Can files be created (ie. have create: true) in READ mode or only WRITE?
7. What's the set of valid mode strings? Currently I've got 'READ' and 'WRITE' but I was thinking maybe there's 'TRUNCATE'?
8. Same as 1. but for openFile requests with create: true.

When #17 is merged, WebDAV and S3 will both use the same filesystem contents for unit tests. Since they also expose identical APIs, this means that a lot of tests can be shared between them. When future providers come along, having a suite of shared tests will be very useful because it gives them a specification straight away without having to write their own.

Requires setting a selected class on the selected element and styling it accordingly

There's a lot of shared code between Amazon S3 and WebDAV. It's highly likely that the other future protocol providers (Samba, SFTP, Git etc) will need to share this code too. It would be nice to have a tool to scaffold providers automatically to speed things up in future.

Files that share lots of code and can be generated:

• readme.md
• package.json
• Gruntfile.js
• ui/auth.html
• ui/auth.js
• ui/style.css

Started a wiki page. We should link to this in the top level readme.

Now that the system is updated to the Chrome's internationalisation framework, we should convert the strings into other languages.

First we need to make sure all the strings are final and the UI is finished though.

Bucket names can be up to 63 characters. Server URLs can be a lot longer. We obviously can't have a sidebar wide enough for this.

Options:

• overflow: scroll. Easy but not good UX.
• Use a truncation library that adds ellipses like dotdotdot. Nicest UX but introduces external dependencies.
• Truncate to a fixed number of characters that will definitely fit. Easy but not as aesthetically pleasing.

There is an issue with some Polymer components that is causing the UI page to not render.
See: Polymer/polymer#704

We currently have a pretty good unit test suite for the two providers, that tests each of the FSP events as well as the wrapper classes and utility functions. However, now that a UI is being added, it would be good to have some tests to verify the overall behaviour of the system.

For regular webapps there are tools like Selenium Webdriver, but I've never used anything like that I don't whether it would work with a Chrome packaged app. We need an environment with the chrome.* APIs for testing (unless we mocked them in the browser?)

Options that won't work

• Zombie: uses JSDOM which doesn't have the features Polymer needs
• Casper: uses Phantom which doesn't have the features Polymer needs
• Selenium WebDriver: Introduces Java dependency
• Dalek: can drive Chrome, but is in developer preview and therefore not suitable for a production app

Still to try

• The Intern
• Sauce Labs

I was unable to find a size property in the WebDAV XML response tree. May be the case that it's necessary to make a GET request for the file and check the body length.

I originally thought that the modularity here was good but really all the files are about 50 lines so they can probably all be combined.

Pros of separate files:

• Easier to instantly jump to what you want in the file list
• Small Browserify output for testing (each test case only includes what it needs (this could be improved with more intelligent browserifying though, ie. each test being required by one meta tests.js that is then loaded into mocha)).

Pros of one file:

• Keeps project structure simpler
• No need to remember to import each one into main.js
• Consistent with WebDAV
• Fewer license headers and 'use strict's.
• One less level of ../ when require()ing

I've been thinking about some of the problems I'm having with the current overall design of the filesystem UI and I think it might be better to take a step back and consider other options before continuing with the current approach.

There are two main problems right now:

• Duplication. Every file system provider will need its own UI, and they will all be incredibly similar. There will be

  • A list of servers/buckets/repos etc in the left hand panel
  • A button to add a new one
  • A button to remove an existing one
  • Some text fields for editing them

  Even when using Polymer components, this is a lot of duplicated effort each time.

• Synchronisation. The list of mounted servers is essentially just a duplicate of the list that already exists in the sidebar of the Files app, and it's a lot of work to ensure that the two stay in sync across additions, edits and removals. This is an area where potential bugs can be introduced that could be easily avoided.

Therefore, I propose that it would be far better to integrate this into the Files app. Have a single, canonical list of mounted file systems on the left sidebar of the file browser as we do already. Then, simply add a button to the file browser (either on the main UI or in the settings) for adding a new provider. Clicking this would bring up a provider-defined HTML page with a single form for entering the required details (bucket and region for S3, URL for WevDAV, etc). These credentials would then be passed into a mount method registered by the provider. Unmounting could then be done as it is already with a button that appears when you hover over a file system in the list, and editing could simply be done inline from the file browser with one more button.

To implement a UI all a provider would need to do is provide the HTML page for entering the configuration and a callback function for responding to a mount request. This is significantly less work than is required with the current setup, and would speed up development for this and all future providers.

This would take a bit longer initially because it would require coordinating with the team that develops Files.app to integrate this, but it would be worth it in the long run, by reducing code duplication, reducing potential areas for bugs, and improving user experience.

I think all Google open source projects are supposed to have them

Since there's all the JSDoc annotations now, we may as well actually run it though JSDoc to generate some HTML documentation and make this available to people who want to contribute to the project.

Could use https://github.com/gotwarlost/istanbul

We switched to Apache 2 but the LICENSE file is still BSD

Currently all the unit tests run in PhantomJS. It works, but it's running a very old Webkit build, which means constantly polyfilling functions for it. So far in the project I have had to polyfill:

• Function.bind
• ArrayBuffer.slice
• Number.isNaN
• The AWS SDK's HTTP request object

And if we want to use the new native ArrayBuffer to string conversion library we will need to Polyfill TextDecoder too.

It's a waste of time constantly trying to work out whether an error is due to a bug in the code or a missing feature in PhantomJS, so we should switch over to running the tests in Chrome, for a consistent environment. The best way to automate this seems to be Karma.

Since S3 is a flat filesystem, and we need to use this to emulate a hierarchical one, there are a few challenges when it comes to write operations:

cc @mtomasz-chromium @jmuk

• How should createDirectory work? S3 has no concept of an empty directory. We could just create an empty file representing the directory's prefix eg /foo/bar, but this is redundant - if you want to write to file /foo/bar/bax.txt when /foo/bar doesn't exist, you can just do that straight away - there's no real directories, just prefixed object keys.
• Same problem as above but with deleteEntry: when the last file with a given prefix is deleted, how do we keep that directory around?
• How should deleteEntry with recursive: true work? Just delete all objects with the prefix?

All these problems could be solved with a local index like the one I initially used, but that still has all the same disadvantages - it needs to be kept in sync, it uses more memory and it makes the code more complicated.

Currently the tests all have to run on different files to avoid conflicts. For example, if you tried to use 1.txt for the read test after using it for the delete test, it would fail because it would have been deleted. If we gave every test a fresh file system the process of adding new tests would be less error prone.

One way to do this would be to copy the filesystem contents over from JS using Mocha's beforeEach hook.

Currently in many places, when an error is encountered, the generic 'FAILED' error code is returned to the Chrome FSP API.

Logic should be added to determine the type of error that has been encountered and select the most relevant error code from this list accordingly.

I've started on this but it's not fully working yet. The API method to use is headBucket, but this has the unwanted side effect of logging a bunch of network errors to the console if the bucket doesn't exist or the user isn't authorised to access it, before the response even arrives to do clean error testing.

It's also currently not correctly returning the error code from the background script to the UI script. I'm using https://developer.chrome.com/extensions/messaging

This is due to the fact that the CSS fill property doesn't work for Polymer SVG icons after running through the Vulcanize tool.

See Polymer/vulcanize#53

The WebDAV test suite has a Mocha before call in its helper/setup script that checks that the server is up before running the tests, and displays a useful error message. The S3 suite does not have this, and therefore simply fails on the first actual test with no explanation why.

This before helper should be included in every script.

Currently only the test files are linted in the CI build. It would be good to add a generic Grunt lint task that runs all 4 linters (tests, main JS, UI JS and the Gruntfile itself) to catch any formatting issues in the CI.