Support for datasets in authenticated S3-compatible private buckets about neuroglancer HOT 6 OPEN

The place to add support would be here:
https://github.com/google/neuroglancer/blob/master/src/neuroglancer/util/special_protocol_request.ts

However, there are a few issues to consider:

• You will need to know the bucket region in order to generate a correct signature. It is a bit tricky to figure it out, see e.g. the approach we take in tensorstore (https://github.com/google/tensorstore/blob/4e82af4392fe4939875d80f9fa5fd01c55beda64/tensorstore/kvstore/s3/s3_endpoint.cc#L211) but here there is the added challenge that it needs to work within the cross-origin request limitations of the browser. Alternatively you could require that the bucket region be specified manually as part of the datasource URL, though that is a bit annoying.
• If you put the access key in the datasource URL itself, then sharing the Neuroglancer URL/state will also share the access key. That may in some cases be desired, but you would need to be careful to use an access key with only the limited privileges that you intend to grant. It would be very easy to accidentally share a more privileged access key. For GCS, I implemented ngauth, which allows neuroglancer users to access private GCS buckets. It requires that you run an ngauth server, which verifies a given user has access to a bucket, and then provides a time-limited restrict authentication token. For S3, I think there may be a way to accomplish something similar, without even the need for a server, by using AWS Cognito combined with suitable S3 access policies (https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_cognito-bucket.html). I expect in many cases that would be preferable to embedding the access token directly in the URL. Nonetheless, it could make sense to still support directly embedding the access token in the URL, and that is ultimately simpler.
• A final issue is that the access token would ideally come at the beginning of the URL, before the bucket name, rather than at the end, so that completion works.

from neuroglancer.