In a deployed service, I don't want to hardcode AWS credentials in my code or even in environment variables. I would like to rely on the IAM role of the machine I'm on.

Is this possible with this library?

It would be nice to have this.

I could do a PR for it if wanted.

Thanks!

I had a ClassNotFoundException using datomic-pro "0.9.5130" and clj-aws-s3 "0.3.10" when I connect the DynamoDB. The datomic uses aws-java-sdk-core 1.8.11 and clj-aws-s3 the 1.7.5.

We've been working to track down a strange issue for a week or two, and believe we have isolated it. My theory is that when S3 generates a 500-level server issue, the AmazonS3Client instance is no longer usable and all requests generate "Stream Closed" exceptions.

I believe the memoization of s3-client as referenced in #3 has made it such that once we get such a server exception, we cannot use clj-aws-s3 without a JVM restart. I don't have a great way to replicate and prove this just yet, however. I'm working on it though.

Presuming so, I see a few possible patches I could submit that fix the issue but remain usuable for the use case described in #3 but would like your input - make memoization optional, create a with-client form of invoking the methods that lets the invoker decide on a client re-use strategy, or wrap calls to most methods in a handler that will regenerate the s3-client in the event of an exception.

Thoughts?

Related clj-time/clj-time#190

I even copied the title :)

When using Java 1.8u60 this exception is thrown when calling aws.sdk.s3/get-object.
It seems to be that Java 1.8u60 requires a later version of joda-time (2.8.1). This problem was fixed in clj-time 0.11.0.

So I forked, changed the dependency to 0.11.0, installed and the error was no more.

(s3/put-object cred "my_bucket" thumb (io/file (str uploads-dir thumb)))

I'd like to make a fully public / web-accessible image

what goes next?

I'm trying to retrieve a large list of files form an s3 bucket. To do so I need to call listNextBatchOfObjects on a ListObject, which is what is returned from listing files. Is this currently possible in this library?

When downloading large objects with get-object, I reliably get "socket is closed". This is caused because the (s3-client cred) is GC'd before my file is done downloading.

See https://forums.aws.amazon.com/thread.jspa?threadID=83326, the reply by Yosuke on Jan 23,2012 3:46 PM for coroboration.

I'm doing

(let [resp (s3/get-object bucket key)]
  (io/copy (-> resp :content) (io/file dest-path))))

My download takes several minutes to finish, and I reliably get the "socket is closed" less than a minute into the download. Replacing (s3/get-object) with its source code, and keeping a reference on the s3-client object fixes the bug.

Just a heads-up that this dependency seems to be broken atm - am assuming this is a problem with the Maven repo. Not sure if/when they'll get around to fixing it.

Is any way to set metadata of an new object when put it into a bucket , or set metadata of an existing object?

A small feature request. To be honest this isn't particularly hard to do directly:

(def s3-client
  ;; This looks for credentials in the standard places.
  (AmazonS3Client. (DefaultAWSCredentialsProviderChain.)))

But it'd be nice if this could be achieved via your s3-client function too so you can combine it the clojurey conventions for specifying the ClientConfiguration.

=> (s3/list-objects cred bucket)

NoClassDefFoundError Could not initialize class com.amazonaws.ClientConfiguration  com.amazonaws.services.s3.AmazonS3Client.<init> (AmazonS3Client.java:365)

Happens on both versions 0.3.3 and 0.3.10 fwiw.

at s3.clj:121

Looks like at least in aws-java-sdk "1.9.33" you can add call .setGeneralProgressListener on PutObjectRequest objects.

1. do you have plans to update clj-aws-s3 to newer aws-sdk?
2. would you consider passing in a progress listener of some kind?

🐪

Looks like it's pretty easy to request byte ranges using the (.setRange ...) method on the GetObjectRequest object.

Not sure what the best way to expose it is. In the meantime I'll be doing the interop myself.

I need to get hold of an object for a particular version. I can provide a patch if necessary.

Related: aws/aws-sdk-java#421

The jar in clojars has a dependency of aws sdk 1.4.2.1. Please update.

The current aws sdk version is 1.7.1, Is there any reason not to upgrade to it?

I have a simple function to post a file in a bucket

(defn post-pic
  "Uploads a picture to S3"
  [file]
  (let [{:keys [filename tempfile]} file]
    (s3/put-object cred my-bucket filename  tempfile)))

It works like a charm, however, it throws everytime an IllegalArgumentException saying that

No implementation of method: :render of protocol: #'compojure.response/Renderable found for class: com.amazonaws.services.s3.model.PutObjectResult

I'm getting com.amazonaws.services.s3.model.AmazonS3Exception exception with this description:

Status Code: 400, AWS Service: Amazon S3, AWS Request ID: 2A700E5BB5693E09, AWS Error Code: InvalidRequest, AWS Error Message: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

The only operation that I can execute successfully is:

(s3/list-buckets cred)

That works, but only if I don't specify :endpoint in by cred's. If I add :endpoint "s3-eu-central-1.amazonaws.com" to my cred's, then the list-buckets gives that same exception.

Any ideas what could be wrong?

i need to create a hierarchy of date folders..

Edit: nevermind i am dumb i nad a leading / as soon as i removed it worked fine and created the folder

Hi -- great lib!,

However, been struggling with put-object and a default permission to be :all-user :read but keep getting error.
I think this is a common use case and an extra example to put-object with permission will be helpful.

I'm using fake-s3 and openstack as my s3 providers & i'm currently writing patch to add function to change default domain of s3 server.

Changing host is simple just (.setEndpoint client "http://new-host") and works well.

But i'm little stuck. How to change PathStyleAccess?

I'm not very familiar with Java and it's documentation is very confusing. Changing PathStyleAccess is important because none of my s3 server dont support accessing buckets as virtualhost .s3.localhost.
Here is documentation: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/S3ClientOptions.html