Support for writing to Google Cloud Storage buckets about mountpoint-s3 HOT 12 OPEN

Just to add that several of our other SDKs already send Content-Length: 0 for CreateMultipartUpload (I tested boto3), so I'd expect adding this to work fine in terms of S3 compatibility. We'd be happy to merge the fix once it's in the CRT library that @arsh pointed to.

from mountpoint-s3.

Hi, thanks for the feature request. We do not support third-party services. However, we are happy to make changes and/or welcome contributions, provided that they are not specific to another service and, most importantly, do not require separate testing.

from mountpoint-s3.

Hey @arsh thanks for your reply.

It sounds like the required change (1) would require changes specific to another service, GCS, and (2) would require separate testing. The wontfix label accords with this.

Makes sense to me. If we really want this functionality it sounds like we should fork 👍

from mountpoint-s3.

One thing that @jamesbornholt pointed out is that this might be a small change in the underlying S3 client Mountpoint uses[1]. Specifically, to put the object in S3, we're creating a multi-part upload and that involves several HTTP requests to create it, upload part, and complete it. It appears that the create multi-part upload is not setting the Content-Length header [2] while the others are.

[1] https://github.com/awslabs/aws-c-s3
[2] https://github.com/awslabs/aws-c-s3/blob/main/source/s3_request_messages.c#L249

from mountpoint-s3.

Yes, sorry, that is possibly the only problem. I should've called this out more obviously in the issue, but it's in the error message:

<p>POST requests require a <code>Content-length</code> header. <ins>That’s all we know.</ins>\n

from mountpoint-s3.

I'll submit a PR to there then, thanks!

from mountpoint-s3.