matsluni / aws-spi-akka-http Goto Github PK

With the change here be5d3c6, the service port map was changed to use the single edge port (4566) used by localstack for all services. The config is described here: https://github.com/localstack/localstack#configurations.

To further clean the ports config this line can be further simplyfied by just return the unique port used by localstack.

Users of this library would normally want to exclude the unused HTTP client libraries brought in by the AWS SDK so they don't interfere with anything.

This should work, right?

"software.amazon.awssdk" % "sns" % AwsSdk2Version excludeAll (
          ExclusionRule("software.amazon.awssdk", "netty-nio-client"),
          ExclusionRule(organization = "io.netty"),
          ExclusionRule("org.apache.httpcomponents", "httpclient")
),

At the moment there's no way to set proxy to AkkaHTTP Client via ClientTransport.httpsProxy(proxyAddress). Unfortunately, there's no way to interact with SDK behind a corp proxy.
Have you thought to add this piece in the builder?
Thanks in advance ☺️

Looks like when Cloudwatch is trying to POST some metrics it fails because the default content-type is set to application/x-www-form-urlencoded.

software.amazon.awssdk.services.cloudwatch.model.CloudWatchException: When Content-Type:application/x-www-form-urlencoded, URL cannot include query-string parameters (after '?'): '/?Action=PutMetricData&Version=2010-08-01&Namespace=...

This is a POST to https://monitoring.us-west-2.amazonaws.com?Action=PutMetricData&Version=2010-08-...

Seems to fall through this code with the final None condition:

private def contentTypeHeaderToContentType(headers: List[HttpHeader]): ContentType = {
    headers.find(_.lowercaseName() == "content-type").map(_.value()) match {
      case Some("application/x-amz-json-1.0") => AkkaHttpClient.xAmzJson
      case Some("application/x-amz-json-1.1") => AkkaHttpClient.xAmzJson11
      case Some("application/x-www-form-urlencoded; charset=UTF-8") => AkkaHttpClient.formUrlEncoded
      case Some("application/x-www-form-urlencoded") => AkkaHttpClient.formUrlEncoded
      case Some("application/xml") => AkkaHttpClient.applicationXml
      case Some(s) => tryCreateCustomContentType(s)
      case None => AkkaHttpClient.formUrlEncoded
    }
  }

Currently, this library uses the default ways to configure akka-http, either via application.conf or via the optional connectionPoolSettings. The AWS Java V2 SDK has its own way to configure the http clients, additionally with AWS Service specific defaults.

In #226 (comment) it was discussed to use the AWS SDK specific ways to configure the underlying akka-http client:

I understand right now that configuration is done via the normal methods of configuration i.e typesafe config library, but what are your thoughts about changing that to use the attributeMap? Asking because I see some specific clients configuring specific options like KinesisHttpConfigurationOptions and you can find other examples here
Maybe we should honor those...

If we would allow this, we would need to make sure, that this happens in a backward compatible way. So, that people, which are already use the optional connectionPoolSettings are not surprised with different configs coming from the new AWS specific config.

Add tests for failure cases, probably needs Wiremock.

See #6 why this is needed.

http clients in aws sdk v2 implements a ProxyConfiguraion object which is consumed by SdkHttpClient to obtain proxy settings.

This library follows a different approach with withConnectionPoolSettings method.
What are your thoughts on unifying the behavior by having a proxyConfiguration method on the AkkaHttpClient.Builder.
AkkaHttpClient then have to handle creation of ConnectionPoolSettings with the ProxyConfiguraion

We may have to expose some of the ConnectionPoolSettings via AkkaHttpClient.Builder just like NettyNioAsyncHttpClient.Builder in the aws SDK in lie of loosing full access to the ConnectionPoolSettings

What are your thoughts on this?

If an actor system isn't configured via withActorSystem this library creates its own actor system which is the best alternative when used via reflection.

It might be useful to complement the AkkaHttpClient with a builder method that expects an implicit actor system. I can't come up with a good name right now, though.

getting this

sdkclientexception "Unable to execute HTTP request: Service returned null 'Content-Length'."

I noticed that AWS java sdk supports http/2, but this client is hardcoded to use http1.1

Any thoughts about adding support for it? I can try to open a PR...

The following dependency libraries are yet to be available for Scala 2.13:

"org.elasticmq" %% "elasticmq-server" % "0.14.1" % "test"
"io.findify"    %% "s3mock"           % "0.2.5"  % "test"

With the current implementation a shared connection pool per ActorSystem is used for all requests.

Its probably better to have a dedicated connection pool per Host.
See akka/alpakka#1958 and akka/alpakka#1983 for similar issue and PR.

Would it be possible to consider producing a Pekko equivalent of this lib?

Pekko HTTP is not yet released but snapshot jars can be found at https://repository.apache.org/content/groups/snapshots/org/apache/pekko/

Pekko Connectors is our open source fork of Alpakka and Alpakka contains a connector for aws-spi-akka-http and our plan is to produce Pekko versions of core Alpakka connectors.

It'd be nice to somehow measure the performance compared to the Netty connector.

There seems to be a bug with the Async client due to the encoding of the username as part of the request.

Steps to reproduce:

• (OPTIONAL) Register a user to quicksight via email - eg. "[email protected]"
• Instantiate a QuickSightAsyncClient
• call describeUser() with: a valid "awsAccountId", "namespace" being "default" and pass an email (eg. "[email protected]") as "userName" (regardless of whether it exists or not).
• Await for the future completion

StatusQuo: An exception is thrown "QuickSightException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
ExpectedResult: Successful future if user exists, or "ResourceNotFoundException" if user does not exist.

val quicksight: QuickSightAsyncClient = AkkaHttpClient
    .builder()
    .withActorSystem(system)
    .withConnectionPoolSettings(
      ConnectionPoolSettings(system)
        .withMaxConnections(15)
        .withIdleTimeout(15.seconds)
    )
quicksight
      .describeUser(_.awsAccountId("ACCOUNT_ID")
        .userName("USER_NAME_AS_EMAIL")
        .namespace("default"))

The same applies to the deleteUser, but not to the registerUser.
It seems to be that some encoding is happening to the username (which is part of the path in delete/describe) and when that changes the username (as is the case with an email), it leads to the exception in question.

If a simple username is used (eg. "myname"), it will work fine.

PS: I initially file this in the sdk repo. Feel free to check it out for further information.

In Release 1.0.1 akka is updated to version 2.8.4. But this akka version uses a business license https://github.com/akka/akka/blob/v2.8.4/LICENSE which means this project can't be licensed with the apache license 2.0 as of version 1.0.1.

Could you please downgrade akka to 2.6.21?

Please upgrade Akka to the latest otherwise dependency is flagged by OWASP checker https://github.com/jeremylong/dependency-check-gradle

http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2017-1000034

Akka versions <=2.4.16 and 2.5-M1 are vulnerable to a java deserialization attack in its Remoting component resulting in remote code execution in the context of the ActorSystem.
CWE-502 Deserialization of Untrusted Data

CVSSv2:
Base Score: HIGH (9.3)
Vector: /AV:N/AC:M/Au:N/C:C/I:C/A:C
CVSSv3:
Base Score: HIGH (8.1)
Vector: /AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H

References:
CONFIRM - http://doc.akka.io/docs/akka/2.4/security/2017-02-10-java-serialization.html
Vulnerable Software & Versions: (show all)

cpe:2.3:a:akka:akka:::::::: versions up to (including) 2.4.16
...

I think this was solved with the last PR, but, I'm getting this error when I try to create a queue using the 0.0.4. Is there anything left before releasing 0.5 (also for 2.13.RC1)?

it'll help to keep the AWS SDK version up to date: https://github.com/fthomas/scala-steward

Update dependencies to be able to supprt Scala 2.13-RC{1,2}

Do you think it'd be good to add more tests and use @localstack for the mocks?