Add Threadsafety for x-bunq-server-signature about sdk_python HOT 14 CLOSED

D'oh! Look at this:

{
	"Error": [{
		"error_description": "Too many requests. You can do a maximum of 3 GET call per 3 second to this endpoint.",
		"error_description_translated": "Too many requests. You can do a maximum of 3 GET call per 3 second to this endpoint."
	}]
} {
	'Date': 'Mon, 07 Aug 2017 11:42:00 GMT',
	'Server': 'Apache',
	'X-Bunq-Client-Response-Id': 'b8b95ad5-170b-4657-9ca9-41cbe06f195f',
	'X-Bunq-Client-Request-Id': '',
	'X-Frame-Options': 'SAMEORIGIN',
	'Transfer-Encoding': 'chunked',
	'Content-Type': 'application/json',
	'Strict-Transport-Security': 'max-age=31536000;'
}

Multi-threading is too fast! 🤣

Ok, I guess I have to revisit my code in order to minimise the GET requests or live with the slightly slower performance of my script. Either way, the error should probably be thrown before the verification is done, or the server should give a signature for the error response as well.

from sdk_python.

Well thanks anyway guys! I guess this topic can be closed now.

from sdk_python.

@PJUllrich here we go! I wonder though why it goes through the assertion... Ahh wait! You've got the old version where validation is done before asserting for errors :). Hold on, we'll do a proper release and you'll be able to include the newest version where this nastiness is fixed.

from sdk_python.

@PJUllrich The most common reason of x-bunq-server-signature is when an error is returned by API and somehow missed by the self._assert_response_success(response) method.

Could you please add this:

print(response.content.decode())

to line 103 of our client.py and send the output here?

from sdk_python.

I added the requested print statement in client.py and added another one in security.py between lines 249 - 250 (just before the Exception occurs). The print statement read:

print(f'From security.py / validate_response - {headers}')

I added the outputs for

1. A response from the server that lead to the Exception and
2. A response for a payment made at the 'same' time that didn't lead to an Exception:

Since the output was quite long, I added it to PasteBin and redacted sensitive information:

1. https://pastebin.com/qu6qKWTJ
2. https://pastebin.com/hWtMmNVx

As you can see, in the headers of the 1. (exception-causing) response, the x-bunq-server-signature is indeed missing. In the 2. response, said signature is available.

from sdk_python.

@PJUllrich I am interested in the response body of the request where validation fails... These outputs include everything but that body :(

from sdk_python.

That one was not printed since the verification failed I figured. Let me check once again.

from sdk_python.

@PJUllrich that's why I asked adding the print statement before the verification! :)

from sdk_python.

I added it there, but the output was humongous! I thought I copied everything necessary, but apparently I missed that one response body. I'll give it another try.

Also, please post the lines of code before which you want to have the print statement. You said line 103 and I put it there, but now that I inspect the code that I have, it's actually behind the verification. I will now put the print statement before this code snippet:

if self._api_context.installation_context is not None:
    security.validate_response(
        self._api_context.installation_context.public_key_server,
        response.status_code,
        response.content,
        response.headers
    )

from sdk_python.

@PJUllrich in the newest develop version of sdk_python, the lines go as:

    response = requests.request(
        method,
        self._get_uri_full(uri_relative),
        data=request_bytes,
        headers=all_headers
    )
    # line 103
    self._assert_response_success(response)

    if self._api_context.installation_context is not None:
        security.validate_response(
            self._api_context.installation_context.public_key_server,
            response.status_code,
            response.content,
            response.headers
        )

from sdk_python.

Aha, @dnl-blkv he is using the SDK published on PyPi. The sdk uploaded on PyPi is not up to date with the latest version of development...

@PJUllrich could you checkout the dev branch and place the sdk in your source code ? It is not possible to upload dev branch to PyPi as of I need to supply a newer version number, according to my knowledge may need to double check this.

from sdk_python.

@PJUllrich xD. @dnl-blkv might be an idea to support multi threads for python ?

from sdk_python.

@OGKevin probably yes, but a bit later

from sdk_python.

@PJUllrich However, our rate limiting remains non-friendly to concurrence. For now, you can workaround it by creating a shared registry storing, for every endpoint, times of last 3 GET calls, last 5 POST calls and last 2 PUT calls and checking against those if the call can already be made (or sleeping otherwise). Here's the page where the limits are specified: https://doc.bunq.com/api/1/page/errors

from sdk_python.