disruptek / atoz Goto Github PK

Hi @disruptek

Currently the library uses os.getEnv("AWS_ACCESS_KEY_ID", AWS_ACCESS_KEY_ID) for multiple procs. Any specific reason for this? I would like to be able to specify the key, secret and region without accessing the environmental variables.

If that's okay with you, I can prepare a PR. Suggestions:

• Duplicated atozSign() proc for specific values
• Specific signing proc, atozSignGetValues(key="", secret="", region=""), which checks the params for values, otherwise returns the environmental
• Expand current atozSign() proc's params with key="", secret="", region="" - if empty then use environmental

Hi @disruptek

I'm testing the various API's and now I have come to testing the library within a multi threaded universe.

When enabling --threads:on the compilation fails due to gcsafe. In atoz we have many global vars, but I can manually convert them into proc's, e.g.:

var headObject* = Call_HeadObject_21626621(name: "headObject", xxxxx

But we are still relying on the rest-package with the Recallable, which also blocks the usage of --threads:on.

All of this can be "avoided" by using a {.gcsafe.}:-block - but whats the impact? Do you have an answer, or can you point me in a direction?

{.gcsafe.}:
    let request = headObject.call(%*{"Bucket": bucket, "Key": key}, query=nil, header=nil, formData=nil, body=nil)

result = await request.issueRequest()

Hi and thanks for a great library!

I tried the SNS/SQS examples from the README file and they work fine. However, when interacting with the DynamoDB API I ran into some issues. First, I tried the following code:

import asyncdispatch, httpclient, httpcore, json
import atoz/dynamodb_20120810

let response = waitfor describeTable.call(%* {"TableName": "my-table"}).issueRequest()
echo waitfor response.body

This results in the following error: Error: unhandled exception: /home/aleksi/.nimble/pkgs/atoz-2602.0.0/atoz/dynamodb_20120810.nim(1904, 10) header != nil' header argument is necessary due to required 'X-Amz-Target' field [AssertionError]. So I went ahead and added the header:

let
  headers: JsonNode = %* {"X-Amz-Target": "DynamoDB_20120810.DescribeTable"}
  response = waitfor describeTable.call(nil, nil, headers, nil, %* {"TableName": "my-table"}).issueRequest()
echo waitfor response.body

Now the request goes through but results in a HTTP 400 response: {"__type":"com.amazon.coral.service#SerializationException"}.

Would you have any idea on what's going on here? Do we need to manually include headers or something?

I'm running atoz version 2602.0.0.

I came across another omission/bug while deploying my code to AWS Lambda. Basically the following code works locally but fails with a UnrecognizedClientException: The security token included in the request is invalid error when running on Lambda:

import asyncdispatch, atoz/dynamodb_20120810, httpclient, json

let
  body = %* { "TableName": "my-table", ... }
  response = await query.call(body).issueRequest()

Turns out Lambda uses Temporary Security Credentials which, in addition to the usual access key id and secret, includes a session token which should be added to the request as either a header or query parameter. Modifying the above example we get this piece of code that now works on Lambda as well:

let
  body = %* { "TableName": "my-table", ... }
  headers = %* {"X-Amz-Security-Token": os.getEnv("AWS_SESSION_TOKEN")}
  response = await query.call(nil, nil, headers, nil, body).issueRequest()

So I was thinking it might make sense for atoz to include the X-Amz-Security-Token header automatically if the AWS_SESSION_TOKEN env variable is defined. This is what the NodeJS and Python AWS SDKs do too, to my knowledge.

What do you think?

Hi, again.

I found a possible bug in Dynamodb 20120810 scan.
When I call it, body of Recallable is empty.

  let
    path: JsonNode = nil
    query: JsonNode = nil
    header: JsonNode = nil
    formData: JsonNode = nil
    body = %* {"TableName": "ReSSt"}
    request = scan.call(path, query, header, formData, body)
    response = waitfor request.issueRequest()
  echo "payload ", request.body # is empty
  echo "status ", response.status # error 400 invalid request

The problems is that atozHook is expecting a jString but scan.call expects body as jObject

Hello,

in the readme I read that this project relies heavily on the OpenAPI code generator. For that, I would assume, an openapi-yml/json definition is used. Where can I get that openapi definition? Does amazon provide one for AWS? This would help to extend this project by missing services.

It looks like all the .nim files in src are being stored in large file format, which means nimble install doesn't work and neither does cloning the repo.

None of the files look like they're big enough to require that, so I'm not sure what happened.

the documentation for the putObject method against the S3 endpoint shows that files, which should be uploaded to S3, need to be placed plain as body in the PUT-request and not a json:
http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectPUT.html

But the implementation asks for JsonNode (of JObject to be concrete) instead of file, which results in empty files in AWS S3. I know what I need to change to get it working, but the code is automatically generated, right? So adding it via a pull request wouldn't be helpful.

Assuming my code looks like the following:

var fileContent =readAll(filename.open)
var fileAsJObject = newJObject()
fileAsJObject.add(key="file",val=newJString(fileContent))

var resp = putObject.call(Bucket="order", Key="customerXYZ/pizza.txt", body=fileAsJObject).retried

then my workaround in s3_20060301 is the following:

• change line 8660: result.add "body", body
• to: result.add "body", body.getOrDefault(key="file")

The cleaner way would be of course to pass a file directly, but I don't see on which base the nim code will be generated.

Defect is probably in swagger definition, but we should work around it.

      "us-west-2": "s3-us-west-2.amazonaws.com",
      "eu-west-2": "s3.eu-west-2.amazonaws.com",