Currently, when
DelegatedTokenCredentials(ApplicationTokenCredentials applicationCredentials, String redirectUrl, String authorizationCode)
here:Constructor is called, the applicationCredentials is not set.

This is the missing line : https://github.com/Azure/autorest-clientruntime-for-java/blob/master/azure-client-authentication/src/main/java/com/microsoft/azure/credentials/DelegatedTokenCredentials.java#L49

Due to the applicationCredentials not being present we get a null pointer : https://github.com/Azure/autorest-clientruntime-for-java/blob/master/azure-client-authentication/src/main/java/com/microsoft/azure/credentials/DelegatedTokenCredentials.java#L169

The DelegatedTokenCredentials(ApplicationTokenCredentials applicationCredentials, String redirectUrl, String authorizationCode) constructor should ideally set authorization code and then call : DelegatedTokenCredentials(ApplicationTokenCredentials applicationCredentials, String redirectUrl)

Our application detects and rethrows this exception while processing some requests using the Azure Java SDK. Narrowing it down reveals that some classes in autorest-clientruntime-for-java should be marked "Serializable".

org.springframework.remoting.RemoteAccessException: Could not access HTTP invoker remote service at [https://127.0.0.2:8443/dt-remote-invoker/remoting/RemotingService]; nested exception is java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: com.microsoft.azure.CloudError

We created a local build and the exception was gone. I have a forked repo containing the fix, which patches the CloudError class and some others.

Not a super high priority, but it might be worth setting up an Azure Container Instance with httpbin to use instead of hammering httpbin.org every time we run tests.

https://hub.docker.com/r/kennethreitz/httpbin/

RestProxy handles the encoded = true argument to PathParam and QueryParam by calling URLEncoder.encode. We may have problems getting the right behavior with both path parameters and query parameters because they're encoded differently and URLEncoder can't tell whether the thing you're encoding is a query argument or a path component.

The code generator should generate Rx overloads of the following types when a BodyParam with a File type is present. Annotations omitted for brevity.

interface MyService {
    Single<RestResponse<Headers, Flowable<Buffer>>> uploadFileViaBody(FileSegment segment);
    Single<RestResponse<Headers, Flowable<Buffer>>> uploadFileViaBody(Flowable<Buffer> flowable);
    Single<RestResponse<Headers, Flowable<Buffer>>> uploadFileViaBody(Buffer buffer);
}

where Buffer is either byte[] or java.nio.ByteBuffer, whichever we determine is better for performance.

The class ServiceResponseCallback.java (and the empty) were removed in this commit, and indeed azure-client-runtime:1.0.0-beta3 do not have these classes. The latest nightly builds from Autorest.exe are currently generating source code which references this class. Were these classes removed intentionally?

7814d4f?diff=unified

Right now we have the @HostParam attribute in the runtime. I'm working on fixing the autorest.java tests and the runtime so that HostParam is used correctly.

Consider the following test Swagger which has multiple host parameters.
https://github.com/Azure/autorest.testserver/blob/master/swagger/custom-baseUrl.json#L12

Right now we generate just one synthetic ParameterJv and use string concatenation followed by parsing to provide the values for all these parameters. This gen code is designed to handle the x-ms-parameterized-host in the Swagger linked above:

public Single<Void> getEmptyAsync(String accountName) {
    if (accountName == null) {
        throw new IllegalArgumentException("Parameter accountName is required and cannot be null.");
    }
    if (this.client.host() == null) {
        throw new IllegalArgumentException("Parameter this.client.host() is required and cannot be null.");
    }
    String parameterizedHost = Joiner.on(", ").join("{accountName}", accountName, "{host}", this.client.host());
    return service.getEmpty(parameterizedHost);
}

I haven't drilled down to the bottom of it, but I think RestProxy might not be dealing with these compound host parameters correctly, and this might be an opportunity to fix the specification and code generator to do this without convoluted string manipulation in Java.

After much discussion and weighing of alternatives, we've settled on the usage of Flowable streams which produce unpooled ByteBuffers. This allows use cases like memory mapped files and uploading views of underlying buffers without copying.

If developers don't like the ServiceResponseBuilder, JacksonMapperAdapter, they should be able to plugin their own implementation in RestClient.

Right now we have special behavior for paging operations. The gist is that we have a simple listSinglePage variant, and an Observable variant which wraps the idea of getting a single page and recursively uses the next page link to get the next page.

It would probably cut down on our gen code a bit to have some special case for when Observable<Page<Entity>> is specified as a return type in AzureProxy where it would handle getting the nextPageLink out and performing the request for each successive page. Does that seem justified?

The addQueryParameter method should overwrite existing parameters with the same key instead of duplicating them (which the method looks to do now since it will just append the supplied key and value). Possibly rename the method to setQueryParameter to make this behavior clear.

Side note: Should the UrlBuilder class be final?

Hello guys.
In the method ApplicationTokenCredentials::acquireAccessToken thread pool is created on each request.
Is it possible to use the same thread pool? Maybe singleton?

Hi,

According to VulnDB scan, OkHttp 3.3.1 library contains a severe security risk:

OkHttp Non-ASCII ETag Header Handling Remote DoS

OkHttp contains a flaw that is triggered during the handling of non-ASCII ETag headers. This may allow a remote attacker to crash a process linked against the library.

Version 3.4.0 is reported not to contain this vulnerability, also the latest version is considered safe.
Please consider updating the OkHttp library dependency, as the vulnerability affects clients using autorest-clientruntime-for-java.

Thank you.

The autorest.java test methods getMbcs and putMbcs fail specifically on the Windows command line, but pass on Mac and Linux, and pass in IDEs on all platforms. This is probably related to the handling of character encodings somewhere in the runtime.

https://github.com/Azure/autorest.java/blob/b74a063fa3aebeaa0129d91a77d9c69904aba139/test/vanilla/src/test/java/fixtures/bodystring/StringOperationsTests.java#L64

Following are the List method variants we are generating today when the list operation is marked as "x-ms-pagable".

Observable<ServiceResponse<Page<T>>> listSinglePageAsync()
	enumerate just the first page (ONE)

Observable<ServiceResponse<Page<T>>> listNextSinglePageAsync(String nextLink)
 	Enumerate just ONE single page identified by the given nextLink.

Observable<ServiceResponse<Page<T>>> listWithServiceResponseAsync()
	Enumerate ALL the pages from the begining.

Observable<ServiceResponse<Page<T>>> listNextWithServiceResponseAsync(final String nextPageLink)
	Enumerate ALL the pages starting from the one with 'nextPageLink'.

Observable<Page<T>> listAsync()
	Enumerate ALL the pages from the begining.

Observable<Page<T>> listNextAsync(final String nextPageLink)
	Enumerate ALL the pages starting from the one with 'nextPageLink'. 

ServiceFuture<List<T>> listAsync(final ListOperationCallback<T> serviceCallback)
        Enumerate ALL the pages from the begining.

ServiceFuture<List<T>> listNextAsync(final String nextPageLink, 
  final ServiceFuture<List<T>> serviceFuture, 
  final ListOperationCallback<T> serviceCallback) 
       Enumerate ALL the pages starting from the one with 'nextPageLink'. 

And the sync method is:

PagedList<T> list();

PagedList can be created from the async versions as:

        PagedList<T> list = return new PagedList<T>(listSinglePageAsync().toBlocking().single()) {
            @Override
            public Page<T> nextPage(String nextPageLink) {
                return listNextSinglePageAsync(nextPageLink).toBlocking().single().body();
            }
        }; 

In the new runtime we need to decide on the variants we are going to support.

When getting the subscriptionId from the Azure class in:
https://github.com/Azure/azure-sdk-for-java/blob/master/azure/src/main/java/com/microsoft/azure/management/Azure.java#L430

The stack creates an AuthenticationContext ignoring the proxy information specified in the Azure/restClient/httpClient/proxy in:
https://github.com/Azure/autorest-clientruntime-for-java/blob/master/azure-client-authentication/src/main/java/com/microsoft/azure/credentials/ApplicationTokenCredentials.java#L239

• Delete AsyncInputStream.java
• Migrate factory methods to FlowableUtil

Since Retrofit 2.0.2 takes dependency on newest OkHttp and Jackson, we should be able to get a consistent dependency tree.

[ERROR]
Dependency convergence error for com.squareup.okhttp3:okhttp:3.1.1 paths to dependency are:
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.squareup.okhttp3:logging-interceptor:3.1.1
+-com.squareup.okhttp3:okhttp:3.1.1
and
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.squareup.retrofit2:retrofit:2.0.0-beta4
+-com.squareup.okhttp3:okhttp:3.0.1
and
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.squareup.okhttp3:okhttp:3.2.0
and
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.squareup.okhttp3:okhttp-urlconnection:3.1.1
+-com.squareup.okhttp3:okhttp:3.1.1

[ERROR]
Dependency convergence error for com.fasterxml.jackson.core:jackson-databind:2.7
.1 paths to dependency are:
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.fasterxml.jackson.core:jackson-databind:2.7.1
and
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.fasterxml.jackson.datatype:jackson-datatype-joda:2.7.1
+-com.fasterxml.jackson.core:jackson-databind:2.7.1
and
+-com.microsoft.azure:BatchHelloworld:1.0-SNAPSHOT
+-com.microsoft.azure:azure-client-runtime:1.0.0-20160330.185245-26
+-com.microsoft.rest:client-runtime:1.0.0-SNAPSHOT
+-com.squareup.retrofit2:converter-jackson:2.0.0-beta4
+-com.fasterxml.jackson.core:jackson-databind:2.4.3

Following error shows, when sign in azure mooncake with service principal file.
{"error_description":"AADSTS50001: The application named https://management.core.windows.net/ was not found in the tenant named 954ddad8-66d7-47a8-8f9f-1316152d9587. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.\r\nTrace ID: 03a0e526-877c-48ed-8723-053777be0a00\r\nCorrelation ID: 9dd42f78-7fac-44cf-b80c-84d933df4345\r\nTimestamp: 2017-10-13 02:21:29Z","error":"invalid_resource"}
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.microsoft.azure.credentials.ApplicationTokenCredentials.acquireAccessToken(ApplicationTokenCredentials.java:145)

After investigation the issue, the root cause is, in https://github.com/Azure/autorest-clientruntime-for-java/blob/v1.1.0/azure-client-runtime/src/main/java/com/microsoft/azure/credentials/AzureTokenCredentials.java#L45 it uses environment.activeDirectoryResourceId() as the request resource. However, when loading service principal in https://github.com/Azure/autorest-clientruntime-for-java/blob/v1.1.0/azure-client-runtime/src/main/java/com/microsoft/azure/credentials/AzureTokenCredentials.java#L45 it only override the active directory url instead of the activeDirectoryResourceId.

The issue impacts all the mooncake user currently. Please help to fix it as soon as possible

RefreshTokenClient takes an optional Proxy as a constructor argument.

Netty and RxNetty don't appear to provide any convenience feature for sending requests through proxies like OkHttp does. I've searched around a bit and the answers seem to involve "just do it yourself" and "look at one of the projects that did it themselves using Netty".

The base URL of a service depends on the environment indicated by your credentials.
https://github.com/Azure/azure-sdk-for-java/blob/4e25b6c5b289af7e1bb3a619a246b37574a07412/azure-mgmt-storage/src/main/java/com/microsoft/azure/management/storage/implementation/StorageManager.java#L48

https://github.com/Azure/autorest-clientruntime-for-java/blob/master/azure-client-runtime/src/main/java/com/microsoft/azure/AzureEnvironment.java#L35

RestProxy needs to be able to take this baseURL, perhaps at the time that RestProxy.create is called.

We need to support long running operations in the new runtime. There is a need for both e.g. a create method that polls the LRO and a beginCreate method that does not poll the LRO. This may be represented by an additional method in the service interface, where one has a @PollLongRunningOperation annotation and the other does not.

To achieve the correct behavior, it may be necessary to change RestProxy to an instance class and subclass it e.g. AzureRestProxy to call out to specific AzureClient methods for polling the LRO.

There are unreasonable limitations to Jackon's XML serialization. To use Jackson, we'd have to wrap lots and lots of list properties in a class containing a single property. I've played a bit with JAXB, the built in XML serialization, and I think we're going to achieve correct behavior with a more usable object model that way.

It'll be necessary to update JacksonAdapter to use JAXB for XML and update the generator to use JAXB annotations.

FasterXML/jackson-dataformat-xml#192

Users need to use HTTP Transfer-Encoding: Chunked. This will require a little work with NettyClient and might affect the interface to AsyncInputStream.
https://stackoverflow.com/a/10544578/2855742

Expected

Run fine in spark environment

Actual

Dependency conflict running the library in spark environment due to jackson-core and jackson-databind. Spark (2+) uses jackson version 2.6.5 where as this library uses version 2.7.2

Exception below -

01:39:43,346  INFO [Thread-6] ExecuteScript:58 - Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.type.TypeBindings.create(Ljava/lang/Class;[Lcom/fasterxml/jackson/databind/JavaType;)Lcom/fasterxml/jackson/databind/type/TypeBindings;
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.rest.serializer.JacksonAdapter.constructJavaType(JacksonAdapter.java:119)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.rest.serializer.JacksonAdapter.deserialize(JacksonAdapter.java:131)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.rest.ServiceResponseBuilder.buildBody(ServiceResponseBuilder.java:216)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.rest.ServiceResponseBuilder.build(ServiceResponseBuilder.java:110)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.azure.AzureResponseBuilder.build(AzureResponseBuilder.java:56)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.azure.keyvault.KeyVaultClientImpl.getSecretsDelegate(KeyVaultClientImpl.java:3152)
01:39:43,347  INFO [Thread-6] ExecuteScript:58 -        at com.microsoft.azure.keyvault.KeyVaultClientImpl.access$1900(KeyVaultClientImpl.java:92)

We need a scheme where an HTTP request in-flight or an HTTP response partially read can be cancelled. .NET accomplishes this with CancellationToken, and we'll want to find something analogous for Java.

Hey

I'm using Azure ARM Java SDK in the context of elasticsearch and elasticsearch is running with a Security Manager.

I'm now blocked by the JacksonAdapter class and method deserialize which actually calls:

        return (T) serializer().readValue(value, new TypeReference<T>() {
            @Override
            public Type getType() {
                return type;
            }
        });

See https://github.com/Azure/autorest-clientruntime-for-java/blob/master/client-runtime/src/main/java/com/microsoft/rest/serializer/JacksonAdapter.java#L117

Basically I'm getting this error:

java.util.concurrent.ExecutionException: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_60]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) ~[?:1.8.0_60]
	at org.elasticsearch.discovery.zen.ZenDiscovery.pingAndWait(ZenDiscovery.java:1000) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at org.elasticsearch.discovery.zen.ZenDiscovery.findMaster(ZenDiscovery.java:860) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:372) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at org.elasticsearch.discovery.zen.ZenDiscovery.access$3800(ZenDiscovery.java:80) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1176) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-6.0.0-alpha1-SNAPSHOT.jar:6.0.0-alpha1-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_60]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_60]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_60]
	at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_60]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_60]
	at java.lang.Class.checkMemberAccess(Class.java:2348) ~[?:1.8.0_60]
	at java.lang.Class.getEnclosingMethod(Class.java:1037) ~[?:1.8.0_60]
	at sun.reflect.generics.scope.ClassScope.computeEnclosingScope(ClassScope.java:50) ~[?:?]
	at sun.reflect.generics.scope.AbstractScope.getEnclosingScope(AbstractScope.java:78) ~[?:?]
	at sun.reflect.generics.scope.AbstractScope.lookup(AbstractScope.java:96) ~[?:?]
	at sun.reflect.generics.factory.CoreReflectionFactory.findTypeVariable(CoreReflectionFactory.java:110) ~[?:?]
	at sun.reflect.generics.visitor.Reifier.visitTypeVariableSignature(Reifier.java:165) ~[?:?]
	at sun.reflect.generics.tree.TypeVariableSignature.accept(TypeVariableSignature.java:43) ~[?:?]
	at sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Reifier.java:68) ~[?:?]
	at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:138) ~[?:?]
	at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49) ~[?:?]
	at sun.reflect.generics.repository.ClassRepository.getSuperclass(ClassRepository.java:90) ~[?:?]
	at java.lang.Class.getGenericSuperclass(Class.java:777) ~[?:1.8.0_60]
	at com.fasterxml.jackson.core.type.TypeReference.<init>(TypeReference.java:33) ~[jackson-core-2.8.6.jar:2.8.6]
	at com.microsoft.rest.serializer.JacksonMapperAdapter$1.<init>(JacksonMapperAdapter.java:179) ~[?:?]
	at com.microsoft.rest.serializer.JacksonMapperAdapter.deserialize(JacksonMapperAdapter.java:179) ~[?:?]
	at com.microsoft.rest.ServiceResponseBuilder.buildBody(ServiceResponseBuilder.java:289) ~[?:?]
	at com.microsoft.rest.ServiceResponseBuilder.build(ServiceResponseBuilder.java:141) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesInner.listAllDelegate(VirtualMachinesInner.java:1201) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesInner.access$700(VirtualMachinesInner.java:46) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesInner$42.call(VirtualMachinesInner.java:1191) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesInner$42.call(VirtualMachinesInner.java:1187) ~[?:?]
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69) ~[?:?]
	at retrofit2.adapter.rxjava.RxJavaCallAdapterFactory$RequestArbiter.request(RxJavaCallAdapterFactory.java:173) ~[?:?]
	at rx.Subscriber.setProducer(Subscriber.java:211) ~[?:?]
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.setProducer(OnSubscribeMap.java:102) ~[?:?]
	at retrofit2.adapter.rxjava.RxJavaCallAdapterFactory$CallOnSubscribe.call(RxJavaCallAdapterFactory.java:152) ~[?:?]
	at retrofit2.adapter.rxjava.RxJavaCallAdapterFactory$CallOnSubscribe.call(RxJavaCallAdapterFactory.java:138) ~[?:?]
	at rx.Observable.unsafeSubscribe(Observable.java:9861) ~[?:?]
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48) ~[?:?]
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33) ~[?:?]
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) ~[?:?]
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) ~[?:?]
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) ~[?:?]
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) ~[?:?]
	at rx.Observable.subscribe(Observable.java:9957) ~[?:?]
	at rx.Observable.subscribe(Observable.java:9924) ~[?:?]
	at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:445) ~[?:?]
	at rx.observables.BlockingObservable.single(BlockingObservable.java:342) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesInner.listAll(VirtualMachinesInner.java:1113) ~[?:?]
	at com.microsoft.azure.management.compute.implementation.VirtualMachinesImpl.list(VirtualMachinesImpl.java:63) ~[?:?]
	at org.elasticsearch.cloud.azure.arm.AzureManagementServiceImpl.lambda$getVirtualMachines$1(AzureManagementServiceImpl.java:100) ~[?:?]

Is it possible not to require this permission?

permission java.lang.RuntimePermission "accessDeclaredMembers";

There are a few configuration properties on RestClient which are no longer used.

• long readTimeoutMillis
• long connectionTimeoutMillis
• int maxIdleConnections

Which of these should be kept and used, maybe added to HttpClient.Configuration, and which should be deleted?

Also, this may be the time to add longRunningOperationTimeout to RestClient, if we think it's acceptable to put an Azure-specific property on there.

Swagger specifies what response codes are valid for a method and what they mean.

e.g. https://github.com/Azure/azure-rest-api-specs/blob/current/specification/compute/resource-manager/Microsoft.Compute/2017-03-30/compute.json#L494

        "responses": {
          "200": {
            "description": "OK",
            "schema": {
              "$ref": "#/definitions/OperationStatusResponse"
            }
          },
          "202": {
            "description": "Accepted"
          },
          "204": {
            "description": "No Content"
          }
        }

The different response codes can have different return types associated with them. I'm guessing that there's usually one return type or nothing--some kind of in-progress value is given back through a header or something. Right now, we have generated code in azure-sdk-for-java for each service method that validates those response codes and otherwise wraps up the response body in an exception.

Perhaps there should be an annotation on a service method which indicates what response codes are acceptable? It would look something like this:

interface UsagesService {
    @Responses({ 200, 202, 204 })
    @Headers({ "Content-Type: application/json; charset=utf-8" })
    @GET("subscriptions/{subscriptionId}/providers/Microsoft.Storage/usages")
    Observable<PageImpl<UsageInner>> list(
        @PathParam("subscriptionId") String subscriptionId,
        @QueryParam("api-version") String apiVersion,
        @HeaderParam("accept-language") String acceptLanguage,
        @HeaderParam("User-Agent") String userAgent);
}

Does this handle our use case?

http://netty.io/wiki/forked-tomcat-native.html

Netty offers several optional modules which use native libraries for SSL. I've been told by others who've been using Netty that this can improve performance significantly.

This is perhaps an opportunity to solidify our stance on native transports as well as native SSL. It would be nice to just always do the best thing for our users with regard to this. However there are so many ways to obtain native SSL, and there are some bugs that come up that are specific to native transports.

I think the most realistic way forward is probably to just show users what they can put in their pom to cause the appropriate native modules to be used, and that way they can test for themselves if the native modules work in their environment.

Alternatively we might be able to default to automatically including the native modules that are simplest to incorporate and add some build-time property to disable them if users want to do their own customization. Adding a "pom-only" project that they can depend on (client-runtime-native/pom.xml)? might even work.

It looks like v2 of Azure SDK for Java will be a good opportunity to update to RxJava 2. There are bound to be a lot of little tweaks needed in the runtime, generator and management SDK to be compatible. Someone should probably take this on as a dedicated task to make it happen from end to end.

After some examination, it's looking reasonable to have users provide an HttpRequestBody instance themselves which produces a fresh InputStream when called and has an accurate contentLength.

This means we won't require users to read the entire chunk to be uploaded into memory before we begin sending a request. We also won't be tied to byte[] or ByteBuf or any other buffer type for the time being. If a user has a byte[] or ByteBuf they want to upload then they can simply wrap in ByteArrayInputStream or ByteBufInputStream.

RestProxy could observe when a @BodyParam parameter has the type HttpRequestBody and place it directly on the HttpRequest instance that gets sent.

It's confirmed that XML support is needed in the new runtime. We think that it just requires a change to the serializer passed to RestProxy.create, and possibly some small changes to the code generator, but it will have to be borne out in practice.

When receiving an HTTP response with status 403, for instance, AzureProxy produces an exception like the following:

com.microsoft.azure.v2.CloudException: Could not determine a long running operation polling strategy (Status code: 403, Azure-AsyncOperation Header: null, Location Header: null

Some kind of tweak in the logic is needed so that an LRO-related message is only produced when it's more relevant. This would also be a good opportunity to make sure that we only attempt to do LRO-related stuff when the response Content-Type is some kind of JSON.

I'm pretty sure that response headers will have to be reimplemented and some of the overloads will have to be added back to the code generator.

I believe Batch service, for instance, gives back values users need to know in headers, as do methods in Blob Storage. https://docs.microsoft.com/en-us/rest/api/storageservices/get-blob-metadata

Most of the plumbing that made this possible should still be around as far as serialization goes. We just have to designate our ServiceResponseWithHeaders equivalent type and start deserializing headers into the header "entities" that are already being generated.

I'm not sure if Azure team is aware of this feature, but it's particularly relevant to the Azure projects for Java, as there are so many inter-dependencies. It might eliminate the need for the intermediate maven repository for hosting snapshots:
https://blog.gradle.org/introducing-composite-builds

It's still in incubation status, but the AutoRest and client runtimes seem like good candidates to try it out on.

to allow further extensions.

From @xingwu1

I found another bug with building header object.

This is the raw header:

Content-Length: 7
Content-Type: application/octet-stream
Last-Modified: Tue, 14 Mar 2017 00:01:39 GMT
Server: Microsoft-HTTPAPI/2.0
request-id: 34e5b1bf-92d5-4316-90d7-ca25f48ce0ff
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
client-request-id: a1835711-2b52-47ed-9671-ac89e4fd370e
DataServiceVersion: 3.0
ocp-creation-time: Tue, 14 Mar 2017 00:01:39 GMT
ocp-batch-file-isdirectory: False
ocp-batch-file-url: https%3A%2F%2Ftest1.westus.batch.azure.com%2Fjobs%2Fxingwu-Job-Mon-Mar-13-17-01-39-PDT-2017%2Ftasks%2Fmytask%2Ffiles%2Fstdout.txt
Date: Tue, 14 Mar 2017 00:01:53 GMT

As you can see, some header keys are with capitalized letter, and here is the generated header object by autorest:

@JsonProperty(value = "Content-Type")
private String contentType;

It looks good here. However, at the runtime (com.microsoft.rest.ServiceResponseBuilder<T, E extends RestException>):

public ServiceResponseWithHeaders<T, THeader> buildEmptyWithHeaders(Response response, Class headerType) throws IOException {
ServiceResponse bodyResponse = this.buildEmpty(response);
Object headers = this.serializerAdapter.deserialize(this.serializerAdapter.serialize(response.headers()), headerType);
ServiceResponseWithHeaders serviceResponse = new ServiceResponseWithHeaders(headers, bodyResponse.headResponse());
serviceResponse.withBody(bodyResponse.body());
return serviceResponse;
}

The result from serialize is:

{"date":"Mon, 13 Mar 2017 23:48:14 GMT","content-length":"7","server":"Microsoft-HTTPAPI/2.0","ocp-batch-file-isdirectory":"False","ocp-batch-file-url":"https%3A%2F%2Ftest1.westus.batch.azure.com%2Fjobs%2Fxingwu-Job-Mon-Mar-13-16-48-01-PDT-2017%2Ftasks%2Fmytask%2Ffiles%2Fstdout.txt","request-id":"78decce7-5128-4b7e-a7d2-7a3cfca4a3b6","strict-transport-security":"max-age=31536000; includeSubDomains","last-modified":"Mon, 13 Mar 2017 23:48:01 GMT","x-content-type-options":"nosniff","client-request-id":"3c1543b6-892f-4d4c-b22a-de996ab12b3e","content-type":"application/octet-stream","dataserviceversion":"3.0","ocp-creation-time":"Mon, 13 Mar 2017 23:48:01 GMT"}

All the keys are lower case, so deserialize will skip the properties with upcase key.

Right now, I manually change the generated file to use lower case as key, and it works. I am not sure whether it is right fix or not.

Useful for web app, native app scenarios.

I am using this library with the azure-keyvault-java library. I was getting 401 Unauthorized no matter what I did when trying to get secrets from a KeyVault. When I looked into it, it was the AzureTokenCredentials.getToken method at this line. The only way I could override this was to extend ApplicationTokenCredentials, which extends this class and overrode the getToken method there. I think this is a roundabout way to get around the problem in my opinion.

My questions are:

• Is there a reason for marking this method as final in AzureTokenCredentials?
• I am not sure if the trailing slash is a problem for any other resources, could you just remove the trailing slash from this line?

File upload is the obvious case for this. Perhaps request bodies should be ByteBuf with a MIME type denoted somewhere. I believe we expect the largest size for a single request body to be on the order of 10 MB.