azure / azure-uamqp-python Goto Github PK

View Code? Open in Web Editor NEW

54.0 222.0 46.0 5.61 MB

AMQP 1.0 client library for Python

License: MIT License

Shell 1.61% Python 66.23% Dockerfile 1.10% Cython 30.89% PowerShell 0.17%

azure-uamqp-python's Introduction

uAMQP for Python

An AMQP 1.0 client library for Python.

Disclaimer

This repo is no longer in active maintenance and we plan on deprecating it sometime in the first quarter of 2025. The EventHubs & Service Bus client libraries are now using the python based AMQP library which is under active development. If there is interest in using the new python library as standalone, please share your interest in this issue.

uAMQP for Python requires Python 3.6+ starting from v1.5, and Python 2.7 is no longer supported. If Python 2.7 is required, please install uAMQP v1.4.3:

$ pip install uamqp==1.4.3

Installation

Wheels are provided for most major operating systems, so you can install directly with pip:

$ pip install uamqp

If you are running a Linux distro that does not support ManyLinux1 or you need to customize the build based on your system settings and packages, you can install from source:

$ apt-get update
$ apt-get install -y build-essential libssl-dev uuid-dev cmake libcurl4-openssl-dev pkg-config python3-dev python3-pip
$ pip3 install uamqp --no-binary :all:

If you are running Alpine, you can install from source:

$ apk add --update python3 py-pip python3-dev cmake gcc g++ openssl-dev build-base
$ pip3 install uamqp --no-binary :all:

If you are running Red Hat, you can install from source:

$ yum install cmake gcc gcc-c++ make openssl-devel python3-devel
$ pip3 install uamqp --no-binary :all:

Documentation

Reference documentation can be found here: docs.microsoft.com/python/api/uamqp/uamqp.

Developer Setup

In order to run the code directly, the Cython extension will need to be build first.

Pre-requisites

Windows: Setup a build environment.
Linux: Install dependencies as descriped above in the installation instructions.
MacOS: Install cmake using Homebrew:

$ brew install cmake

Building the extension

This project has two C library dependencies. They are vendored in this repository in these versions:

To build, start by creating a virtual environment and installing the required Python packages:

$ python -m venv env
$ env/Scripts/activate
(env)$ pip install -r dev_requirements.txt

Next, run the build command:

$ python setup.py build_ext --inplace

Tests

The tests can be run from within the virtual environment. The extension must be built first using the instructions above.

(env)$ pytest

Provide Feedback

If you encounter any bugs or have suggestions, please file an issue in the Issues section of the project.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-uamqp-python's People

Contributors

Stargazers

Watchers

azure-uamqp-python's Issues

Sample sample_uamqp_receive_simple.py has undefined variable "hostname"

$SUBJECT.

Build fails due to warnings turned into errors by passing -Werror

The build currently fails with gcc-8 on openSUSE Tumbleweed with:

[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c: In function 'encode_float_value':
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:3132:34: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]      uint32_t value_as_uint32 = *((uint32_t*)(void*)&value);
[   34s]                                  ~^~~~~~~~~~~~~~~~~~~~~~~~~
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c: In function 'encode_double_value':
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:3196:34: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]      uint64_t value_as_uint64 = *((uint64_t*)(void*)&value);
[   34s]                                  ~^~~~~~~~~~~~~~~~~~~~~~~~~
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c: In function 'internal_decoder_decode_bytes':
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:5226:23: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]                      *((uint32_t*)&internal_decoder_data->decode_to_value->value.float_value) = 0;
[   34s]                       ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:5239:23: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]                      *((uint64_t*)&internal_decoder_data->decode_to_value->value.double_value) = 0;
[   34s]                       ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:5723:23: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]                      *((uint32_t*)&internal_decoder_data->decode_to_value->value.float_value) += ((uint32_t)buffer[0]) << ((3 - internal_decoder_data->bytes_decoded) * 8);
[   34s]                       ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[   34s] /home/abuild/rpmbuild/BUILD/uamqp-1.1.0/src/vendor/azure-uamqp-c/src/amqpvalue.c:5745:23: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
[   34s]                      *((uint64_t*)&internal_decoder_data->decode_to_value->value.double_value) += ((uint64_t)buffer[0]) << ((7 - internal_decoder_data->bytes_decoded) * 8);
[   34s]                       ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[   36s] cc1: all warnings being treated as errors
[   36s] gmake[2]: *** [CMakeFiles/uamqp.dir/build.make:102: CMakeFiles/uamqp.dir/src/amqpvalue.c.o] Error 1
[   36s] gmake[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/uamqp.dir/all] Error 2
[   36s] gmake: *** [Makefile:141: all] Error 2

This is because multiple cmake files pass -Werror to the compiler:

glaubitz@suse-laptop:~/suse/home:glaubitz:branches:devel:languages:python:azure3/python-uamqp> grep -R Werror *
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/configs/azure_iot_build_rules.cmake:    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/configs/azure_iot_build_rules.cmake:    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/configs/azure_iot_build_rules.cmake:    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/configs/azure_iot_build_rules.cmake:    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/umock-c/CMakeLists.txt:# Build with -Werror
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/umock-c/CMakeLists.txt:    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/umock-c/CMakeLists.txt:    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/umock-c/deps/ctest/CMakeLists.txt:    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/umock-c/deps/ctest/CMakeLists.txt:    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/ctest/CMakeLists.txt:    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
uamqp-1.1.0/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/testtools/ctest/CMakeLists.txt:    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Werror")
glaubitz@suse-laptop:~/suse/home:glaubitz:branches:devel:languages:python:azure3/python-uamqp>

At the time, gcc generates warnings for the code. Thus, in order to fix this, either drop -Werror from the C/CXXFLAGS or fix the warnings.

Incorrect garbage collection if "close()" is not called

See Azure/azure-sdk-for-python#8404 for details

Seg fault when sending message to SB without the with/as statement

Hello everyone,
When I tried to send a message to ServiceBus, I experienced segmentation faults. It turns out I tried to send them like this:

topic_client.get_sender().send(Message(msg))

which causes segfault. I tried it on my Fedora PC, as well as Windows PC of a colleague of mine - same results. It seems that calling the sender like this fixes the issue:

with topic_client.get_sender() as sender:
    sender.send(Message(msg))

I can provide a Valgrind log, which is a bit long, so maybe the log from Windows may be enough:

2019-07-11 16:41:07,442 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.END: 13> to <ConnectionState.START: 0>
2019-07-11 16:41:07,469 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
2019-07-11 16:41:07,678 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
2019-07-11 16:41:07,729 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
2019-07-11 16:41:07,730 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
2019-07-11 16:41:07,783 [INFO ]  Connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
2019-07-11 16:41:07,885 [INFO ]  CBS for connection b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e' completed opening with status: 0
2019-07-11 16:41:07,936 [INFO ]  Token put complete with result: 0, status: 202, description: b'Accepted', connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:07,988 [INFO ]  Message sender b'sender-link-d285ab2d-de04-4638-8232-732ce63df535' state changed from <MessageSenderState.Idle: 0> to <MessageSenderState.Opening: 1> on connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:08,090 [INFO ]  Message sender b'sender-link-d285ab2d-de04-4638-8232-732ce63df535' state changed from <MessageSenderState.Opening: 1> to <MessageSenderState.Open: 2> on connection: b'SBSender-b4d12125-eecc-4f5e-95d5-2100c7ca624e'
2019-07-11 16:41:08,295 [INFO ]  b'Failure: sending socket failed 10038.' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\socketio_win32.c':b'socketio_send':422)
2019-07-11 16:41:08,296 [INFO ]  b'xio_send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'send_chunk':484)
2019-07-11 16:41:08,297 [INFO ]  b'send_chunk failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'internal_send':525)
2019-07-11 16:41:08,298 [INFO ]  b'send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\adapters\\tlsio_schannel.c':b'tlsio_schannel_send':1263)
2019-07-11 16:41:08,298 [INFO ]  b'xio_send failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1190)
2019-07-11 16:41:08,299 [INFO ]  b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)

Steps to reproduce

sbbs.py:

import json

from azure.servicebus import TopicClient, Message
from azure.servicebus.control_client import ServiceBusService

CN = "Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=yyy"
TOPIC = "topic_name"    

if __name__ == "__main__":
    topic_client = TopicClient.from_connection_string(CN, name=TOPIC)
    msg = json.dumps({})

    print("Sending a message 1")
    with topic_client.get_sender() as sender:
        sender.send(Message(msg))

    print("Sending a message 2")
    topic_client.get_sender().send(Message(msg))

Output

~ python3 sbbs.py                                  
Sending a message 1
Sending a message 2
[1]    1603 segmentation fault (core dumped)  python3 sbbs.py

It seems the problem is in the underlying azure-uamqp-c library - thus the segfault. But as a user of the python library, I am posting the issue here - apparently, there's a problem in the way the C lib is being used when not using the with/as statement.

Errors in version 1.2.3 and later

I seem to have found an issue when using the azure-eventhub-1.3.2 library with ‘uamqp’ version 1.2.3 and later. Version 1.2.2 doesn't seem to have the issue. Basically, I quickly get errors when I connect to the EventHub using the following code:

import asyncio
from azure.eventhub import EventHubClient, Sender, EventData, EventHubClientAsync

async def main():
    address = "amqps://.....servicebus.windows.net/...."
    user = ...
    password = ...

    client = EventHubClientAsync(address, debug=False, username=user, password=password)
    sender = client.add_async_sender()
    await client.run_async()

    while True:
        try:
            event_str = "test message"
            event_data = EventData(event_str)
            print('Sending Event data {} to eventhub'.format(event_str))

            await sender.send(event_data)
            await asyncio.sleep(1)

        except Exception as e: 
            print (f"caught excetpion {e}")
            await asyncio.sleep(1)


    await client.stop_async()

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()

This code simply connects to the EventHub, sends a message, and sleeps for a second. The first 2 or 3 messages go through, and then it usually gets disconnected and is not retry-able. You have to disconnect and reconnect again to make it work. The output messages I got were:

> python3 ./eventhub_sender.py
Sending Event data test message to eventhub
Sending Event data test message to eventhub
Sending Event data test message to eventhub
Sending Event data test message to eventhub
caught excetpion Send failed: Message send failed with result: MessageSendResult.Timeout
Sending Event data test message to eventhub
caught excetpion Send failed: Message send failed with result: MessageSendResult.Timeout
Sending Event data test message to eventhub
caught excetpion Send failed: Message send failed with result: MessageSendResult.Timeout
Sending Event data test message to eventhub
caught excetpion Send failed: Message send failed with result: MessageSendResult.Timeout
Sending Event data test message to eventhub
caught excetpion Send failed: Message send failed with result: MessageSendResult.Timeout

I tried using the new Python SDK (azure-eventhub-5.0.0r6) against the 1.2.4 library and it seems fine, but when I use (azure-eventhub-1.3.2) against this, it's breaking. The code changes in version 1.2.3 seems to have broken something in the older version.

Note, I'd like to be using the new Python SDK, but it's still beta and not ready for production. I should be able to move over to these libraries when they are stable. In the meantime, this issue should be resolved.

Best way to capture output from link detach

I have a need to capture output from the detach frame. This is related to issue #10 (since they both output on detach), but this is primarily for reporting a clean message to the user. What's the best way to do this?

Here is an example debug log:

uamqp.sender : Message sender state changed from MessageSenderState.Opening to MessageSenderState.Error
uamqp.c_uamqp : b'<- [DETACH]* {0,true,* {amqp:not-allowed,This capability is not available for Basic Tier IoT Hub. Please visit https://aka.ms/StandardIoTHubUpgrade to learn about\xc2\xa0Standard Tier.,{[com.microsoft:tracking-id:abcd123-G:3-TimeStamp:05/16/2018 00:34:49],[com.microsoft:is-filtered:true]}}}'
uamqp.c_uamqp : b'-> [END]* {* {amqp:session:unattached-handle,}}'

reported performance issues

Context of this issue is using monitor-events via IoT extension.

We have some user reports that are experiencing very high CPU usage spikes (up to 100%) - in this case when no messages are being received. The current implementation uses the ReceiveClientAsync.receive_messages_iter_async to poll messages.

I will add more details to this thread as I receive them (like verifying uAMQP version).

Thank you.

Not able to reconnect after Link Detach seems to end up in a loop

So we are using the Python Eventhub library and after a while the uamqp gets an error as seen in the logs.
When this happens the process take full CPU, this is running in a AKS cluster and is given 1000m and it uses all while it is stuck.

The logging that says Testing: One Cycle Completed is just to see when we run one cycle in our while loop.

Have any idea on if it is the C code that get stuck in an loop?

2018-12-13 21:06:02,470 ehubtokafka.src.eventhubconnector INFO Testing: One cycle completed.
2018-12-13 21:06:03,205 ehubtokafka.src.eventhubconnector INFO Testing: One cycle completed.
2018-12-13 21:06:06,072 uamqp.c_uamqp INFO CBS error occured on connection b'###'.
2018-12-13 21:06:06,122 ehubtokafka.src.eventhubconnector INFO Testing: One cycle completed.
2018-12-13 21:06:06,173 uamqp.receiver INFO Received Link detach event: b'amqp:unknown-error'
Link: b'###'
Description: None
Details: None
Retryable: True
Connection: b'###'
2018-12-13 21:06:06,173 uamqp.receiver INFO Message receiver b'###' state changed from <MessageReceiverState.Open: 2> to <MessageReceiverState.Idle: 0> on connection: b'###'
2018-12-13 21:06:06,174 uamqp.connection INFO Connection b'###' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.CLOSE_RCVD: 10>
2018-12-13 21:06:06,175 uamqp.connection INFO Connection b'###' state changed from <ConnectionState.CLOSE_RCVD: 10> to <ConnectionState.END: 13>
2018-12-13 21:06:06,175 uamqp.connection INFO Received Connection close event: b'amqp:unknown-error'
Connection: b'###'
Description: None
Details: None

Fail to build and use on FreeBSD

It is not possible to build and install uamqp on FreeBSD because of 3 different build errors in the bundled azure-uamqp-c.

Workaround:
export CFLAGS="-DUSE_OPENSSL -D__APPLE__ -DTCP_KEEPINTVL=512 -DTCP_KEEPALIVE=0x0008" && pip install uamqp

Build errors:

USE_OPENSSL is undefined, so "azure_c_shared_utility/tlsio_openssl.h" is not included and build failed on deps/azure-c-shared-utility/adapters/platform_linux.c
Functions destroy_network_interface_descriptions, create_network_interface_description, set_target_network_interface etc from deps/azure-c-shared-utility/adapters/socketio_berkeley.c are linux specific and cannot be built on FreeBSD. But it looks like they, at first, are not required for uamqp, then, same issue exists for macOS, so all these functions are inside #ifndef APPLE #endif. So it is enough to declare APPLE to fix this error. It seems another bsd specific ifndef should be added.
And the last error came from undefined TCP_KEEPINTVL & TCP_KEEPALIVE. TCP_KEEPINTVL is defined in netinet/tcp.h, while TCP_KEEPALIVE does not exist, there is SO_KEEPALIVE

Support for websockets

Required for Azure Event Hub SDK as documented in issue #67.

Incorrect datetime object parsing

As reported in SO issue:
https://stackoverflow.com/questions/55357199/mktime-argument-out-of-range

Should involve a fix consistent with the fix added to Azure Event Hubs SDK.

Investigate the right set of C compilation flags

Right now, the setup.py passes "-g -O0" while CMake is passing "-O2"

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/azure-uamqp-python/src/vendor/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/pal/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/inc -I./src/vendor/azure-uamqp-c/inc -I./src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/pal/linux -I/azure-uamqp-python/env/include -I/usr/include/python3.6m -c uamqp/c_uamqp.c -o build/temp.linux-x86_64-3.6/uamqp/c_uamqp.o -g -O0 -std=gnu99 -fPIC

It's unclear without being a gcc expert in what optimisation levels the lib is actually compiled.

Wheel should be the most possible optimized for performance. We might consider adding a "DEBUG_COMPILE" env variable read by the setup.py to compile in debug mode for easier debugging. That's actually changing this line (necessary, maybe not sufficient):

"-DCMAKE_BUILD_TYPE=Release"

"-DCMAKE_BUILD_TYPE=Debug"

Fails in Python 3.4 and older due to circular dependency between types.py and utils.py

The lines from uamqp import types in utils.py and from uamqp import utils in types.py are causing a circular dependency that doesn't work in Python 3.4 (but seems to be okay in 3.5 and 3.6). It appears this can be fixed by changing the lines to import types and import utils, although I'm not sure if that fix will still work in 3.5 and 3.6.

Error: Couldn't get 'time_to_live'

After updating from 0.1.0.rc1 to 0.1.0 I get the following error when receiving any message from the azure service bus

Traceback (most recent call last):
File "/[...]/lib/python3.6/site-packages/uamqp/receiver.py", line 235, in _message_received
settler=settler)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 92, in init
self._parse_message(message)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 142, in _parse_message
self.header = MessageHeader(header=_header)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 958, in init
self.time_to_live = header.time_to_live
File "./src/header.pyx", line 78, in uamqp.c_uamqp.cHeader.time_to_live.get
File "./src/base.pyx", line 31, in uamqp.c_uamqp.StructBase._value_error
ValueError: Operation failed.
Error: Couldn't get 'time_to_live'.
Exception ignored in: 'uamqp.c_uamqp.on_message_received'
Traceback (most recent call last):
File "/[...]/lib/python3.6/site-packages/uamqp/receiver.py", line 235, in _message_received
settler=settler)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 92, in init
self._parse_message(message)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 142, in _parse_message
self.header = MessageHeader(header=_header)
File "/[...]/lib/python3.6/site-packages/uamqp/message.py", line 958, in init
self.time_to_live = header.time_to_live
File "./src/header.pyx", line 78, in uamqp.c_uamqp.cHeader.time_to_live.get
File "./src/base.pyx", line 31, in uamqp.c_uamqp.StructBase._value_error
ValueError: Operation failed.
Error: Couldn't get 'time_to_live'.

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

When commenting line 958 (self.time_to_live = header.time_to_live) of uamqp/message.py the error disappears, it seems that there is a problem with the code loading the time_to_live from the header (no default value or something?)

Update uamqp to be compatible with latest azure-identity get_token interface

The get_token method in azure-identity will now return both the token string and expiration time information, previously it just return token string.

Add tracing details

Could you please add tracing details in Readme file? This will help troubleshoot issues as we can't take fiddler to track calls.

Wheel support for Alpine

Memory Leak when sending Batch Message to EventHub

The usage of memory keeps growing when sending BatchEventData to EventHub in a long running test. Some of the C objects may not well be freed. Further investigation into the pyx files is needed, especially the message.pyx file.

Multiple text files have the executable bits set

Multiple files contained in the source tree have their executable bits set which triggers warnings when building an RPM package:

[  195s] RPMLINT report:
[  195s] ===============
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/PKG-INFO
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/SOURCES.txt
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/dependency_links.txt
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/not-zip-safe
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/requires.txt
[  197s] python2-uamqp.x86_64: W: script-without-shebang /usr/lib64/python2.7/site-packages/uamqp-1.1.0-py2.7.egg-info/top_level.txt
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/PKG-INFO
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/SOURCES.txt
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/dependency_links.txt
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/not-zip-safe
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/requires.txt
[  197s] python3-uamqp.x86_64: W: script-without-shebang /usr/lib64/python3.7/site-packages/uamqp-1.1.0-py3.7.egg-info/top_level.txt
[  197s] This text file has executable bits set or is located in a path dedicated for
[  197s] executables, but lacks a shebang and cannot thus be executed.  If the file is
[  197s] meant to be an executable script, add the shebang, otherwise remove the
[  197s] executable bits or move the file elsewhere.

and

[  197s] python2-uamqp.x86_64: E: spurious-executable-perm (Badness: 50) /usr/lib64/python2.7/site-packages/uamqp/c_uamqp.c
[  197s] python3-uamqp.x86_64: E: spurious-executable-perm (Badness: 50) /usr/lib64/python3.7/site-packages/uamqp/c_uamqp.c
[  197s] python2-uamqp.x86_64: E: spurious-executable-perm (Badness: 50) /usr/share/doc/packages/python2-uamqp/README.rst
[  197s] python3-uamqp.x86_64: E: spurious-executable-perm (Badness: 50) /usr/share/doc/packages/python3-uamqp/README.rst
[  197s] The file is installed with executable permissions, but was identified as one
[  197s] that probably should not be executable.  Verify if the executable bits are
[  197s] desired, and remove if not. NOTE: example scripts should be packaged under
[  197s] %docdir/examples, which will avoid this warning.

To fix this, just unset the executable bits using chmod ugo-x $FILES. Luckily, git is able to track file permissions.

Missing import in async receiver

Need to import errors here:
https://github.com/Azure/azure-uamqp-python/blob/master/uamqp/async/receiver_async.py#L11

Fix unstable async tests with shared connection

Two tests are intermittently failing on OSX and Linux:

test_event_hubs_shared_connection_async
test_event_hubs_multiple_receiver_async

Connection in an unexpected error state with Solace Broker

Hi there,

I've been trying to use uAMQP for Python with a Solace Standard broker running in a local container - Nothing too complex, just the samples provided in this repository. No success so far, although I have a topic and queue set up. I rose the logging level to DEBUG, and it seems to be some issue related to DETACH frame in link protocol - but I'm out of ideas... any idea what could it possibly be? It would be much appreciated :)

Here's the log:

$ python send.py 
DEBUG:uamqp.c_uamqp:Creating SASL Mechanism 
DEBUG:uamqp.client:Opening client connection.  
DEBUG:uamqp:Initializing platform. 
INFO:uamqp.connection:Connection b'17f16ec1-306e-4046-ab48-8b21cfe8bdc9' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0> 
DEBUG:uamqp.c_uamqp:Wrapping value type: <AMQPType.CompositeType: 22>  
DEBUG:uamqp.c_uamqp:Wrapping value type: <AMQPType.CompositeType: 22>  
INFO:uamqp.sender:Message sender b'sender-link-af42104a-114a-48b1-a97c-b7bcab34c21e' state changed from <MessageSenderState.Idle: 0> to <MessageSenderState.Opening: 1> on  connection: b'17f16ec1-306e-4046-ab48-8b21cfe8bdc9' 
INFO:uamqp.connection:Connection b'17f16ec1-306e-4046-ab48-8b21cfe8bdc9' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0> 
INFO:uamqp.connection:Connection b'17f16ec1-306e-4046-ab48-8b21cfe8bdc9' state changed from <ConnectionState.START: 0> to <ConnectionState.END: 13>  
INFO:uamqp.connection:Connection with ID b'17f16ec1-306e-4046-ab48-8b21cfe8bdc9' unexpectedly in an error state. Closing: False, Error: None
INFO:uamqp.sender:Sender link failed to open - expecting to receive DETACH frame.  
WARNING:uamqp.connection:ConnectionClose('ErrorCodes.UnknownError: Connection in an unexpected error state.') 
Traceback (most recent call last): 
  File "send.py", line 39, in <module>
 uamqp_receive_simple()
  File "send.py", line 35, in uamqp_receive_simple
 cliente.send_message(Message(body=msg_content))  
  File "C:\Python37\lib\site-packages\uamqp\client.py", line 694, in send_message  
 running = self.do_work() 
  File "C:\Python37\lib\site-packages\uamqp\client.py", line 377, in do_work 
 if not self.client_ready(): 
  File "C:\Python37\lib\site-packages\uamqp\client.py", line 362, in client_ready  
 self._connection.work()  
  File "C:\Python37\lib\site-packages\uamqp\connection.py", line 252, in work
 raise self._error  
uamqp.errors.ConnectionClose: ErrorCodes.UnknownError: Connection in an unexpected error state.
DEBUG:uamqp.c_uamqp:Deallocating cMessage
DEBUG:uamqp.c_uamqp:Destroying cMessage  
DEBUG:uamqp.c_uamqp:Deallocating cTarget 
DEBUG:uamqp.c_uamqp:Destroying cTarget
DEBUG:uamqp.c_uamqp:Deallocating 'StringValue' 
DEBUG:uamqp.c_uamqp:Destroying 'StringValue'
DEBUG:uamqp.c_uamqp:Deallocating 'CompositeValue' 
DEBUG:uamqp.c_uamqp:Destroying 'CompositeValue'
DEBUG:uamqp.c_uamqp:Deallocating 'CompositeValue' 
DEBUG:uamqp.c_uamqp:Destroying 'CompositeValue'
DEBUG:uamqp.c_uamqp:Deallocating XIO  
DEBUG:uamqp.c_uamqp:Destroying XIO 
DEBUG:uamqp.c_uamqp:Deallocating SASLMechanism 
DEBUG:uamqp.c_uamqp:Destroying SASLMechanism
DEBUG:uamqp.c_uamqp:Deallocating XIO  
DEBUG:uamqp.c_uamqp:Destroying XIO 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection
INFO:uamqp.c_uamqp:b'send called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)
INFO:uamqp.c_uamqp:b'saslclientio_close called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 
INFO:uamqp.c_uamqp:b'xio_close failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272)
INFO:uamqp.c_uamqp:Unknown connection state changed: 13 to 13 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection
INFO:uamqp.c_uamqp:b'send called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)
INFO:uamqp.c_uamqp:b'saslclientio_close called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 
INFO:uamqp.c_uamqp:b'xio_close failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272)
INFO:uamqp.c_uamqp:Unknown connection state changed: 13 to 13 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection
INFO:uamqp.c_uamqp:b'send called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)
INFO:uamqp.c_uamqp:b'saslclientio_close called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 
INFO:uamqp.c_uamqp:b'xio_close failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272)
INFO:uamqp.c_uamqp:Unknown connection state changed: 13 to 13 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection
INFO:uamqp.c_uamqp:b'send called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)
INFO:uamqp.c_uamqp:b'saslclientio_close called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 
INFO:uamqp.c_uamqp:b'xio_close failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272)
INFO:uamqp.c_uamqp:Unknown connection state changed: 13 to 13 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection
INFO:uamqp.c_uamqp:b'send called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268)
INFO:uamqp.c_uamqp:b'saslclientio_close called while not open' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 
INFO:uamqp.c_uamqp:b'xio_close failed' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272)
INFO:uamqp.c_uamqp:Unknown connection state changed: 13 to 13 
DEBUG:uamqp.c_uamqp:Deallocating Connection 
DEBUG:uamqp.c_uamqp:Destroying Connection

Preferred way to handle amqp link redirects

What is the preferred pattern/mechanism to follow amqp redirect links? For example in the case when the IoT Hub redirects to its event hub endpoint.

Appears to emit no valid protocol header?

I have been trying to run a simple example client against various different AMQP servers from the Apache Software Foundation (Apache ActivemMQ Artemis, Apache Qpid Dispatch Router and Apache c++ broker aka qpidd).

In all cases I am unable to connect. My test client is as follows:

import os
try:
    from urllib import urlparse
except ImportError:
    from urllib.parse import urlparse

import uamqp
from uamqp import authentication


uri = os.environ.get("AMQP_SERVICE_URI") or "amqp://localhost/foo"

def uamqp_receive_simple():
    parsed_uri = urlparse(uri)
    anon_auth = authentication.SASLAnonymous(parsed_uri.hostname, port=5672)

    message = uamqp.receive_message(uri, auth=anon_auth, debug=True)
    print("Received: {}".format(message))

if __name__ == "__main__":
    uamqp_receive_simple()

Viewing the interaction with wireshark, it appears that the client does not emit a valid AMQP protocol header at the start of the stream. I see 16 03 01 00 ab 01 00 00 seemingly every time as the first 8 bytes. The client then seems to spin in a loop, with cpu going to 100%.

If it is useful for testing, you can use:

docker run -it -p 5672:5672 docker.io/gordons/qdrouterd:1.3.0

To start an instance of qpid dispatch router listening on 5672 (no TLS, no auth required).

Add support for HTTP proxy settings

_link_error is not cleaned up when multiples links are sharing session

Describe the bug

When multiples links (sender/receiver) are sharing the same session, if one of these links gets into error state, the uamqp.Session would mark self._link_error:
https://github.com/Azure/azure-uamqp-python/blob/master/uamqp/session.py#L48

However, this behavior blocks pending links to establish the connection. Because in _state_changed of sender/receiver, it will check self._session._link_error and find error is not None -- the error is caused by the last link establishment.
https://github.com/Azure/azure-uamqp-python/blob/master/uamqp/sender.py#L170

The variable session._link_error is not cleaned up when new links want to establish connection.

To Reproduce
Steps to reproduce the behavior:

Using ServiceBus write-only connection string for a queue:

    with client:

        try:
            with client.get_queue_receiver(QUEUE_NAME) as receiver:
                messages = receiver.receive(max_batch_size=1, timeout=1)
        except Exception as e:
            assert isinstance(e, ServiceBusError)

        # hotfix, reset _link_error to None before next link establishment
        # client._connection.auth._session._link_error = None 

        with client.get_queue_sender(QUEUE_NAME) as sender:
            sender.send(Message("test"))
            try:
                sender.send("cat")
            except Exception as e:
                assert isinstance(e, ServiceBusError)

The user_id property for the MessageProperties class appears limited to 8 characters across the wire

For some reason sending user_id with more than 8 characters causes the exceeding characters to get truncated.

For example sending a payload with:

msg_props.user_id = "abcdefghijklmnop/"

On the receiving end the user_id is "abcdefgh"

Keep alive failures

When trying to use keep_alive_interval on a set of servicebus queues (where the queues are configured, but no remote client is yet connected) I am hitting errors and unexpected state changes in the amqp connections. Ultimately the connections fail and need to be recreated. If remote clients are enabled AND message traffic is occurring this does not happen, so there must be some level of timeout causing this.

python 3.6.9
uamqp 1.2.5
azure-servicebus 0.50.2

Example python code snippet:
self.input_queue = self.servicebus.get_queue(self.input_queue_name).get_receiver(keep_alive_interval = 30)
self.output_queue = self.servicebus.get_queue(self.output_queue_name).get_sender(keep_alive_interval = 31)

Resulting logs which show the AMQP state changes and failures attached below (note that there are two senders in this example). The sends work one time then start failing. Anyone have ideas on what might be occurring here?

DEBUG:uamqp.client:Keeping 'ReceiveClient' connection alive.
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
DEBUG:uamqp.client:Keeping 'ReceiveClient' connection alive.
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
INFO:uamqp.connection:Connection b'SBSender-5e63e211-e448-4a16-a869-206a95cb0d94' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.DISCARDING: 12>
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
INFO:uamqp.connection:Connection b'SBSender-1831c47e-538e-4aa1-a9bb-6bb23253efa2' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.DISCARDING: 12>
DEBUG:uamqp.client:Keeping 'ReceiveClient' connection alive.
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
INFO:uamqp.connection:Connection b'SBSender-5e63e211-e448-4a16-a869-206a95cb0d94' state changed from <ConnectionState.DISCARDING: 12> to <ConnectionState.DISCARDING: 12>
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
INFO:uamqp.connection:Connection b'SBSender-1831c47e-538e-4aa1-a9bb-6bb23253efa2' state changed from <ConnectionState.DISCARDING: 12> to <ConnectionState.DISCARDING: 12>
DEBUG:uamqp.client:Keeping 'ReceiveClient' connection alive.
DEBUG:uamqp.client:Keeping 'SendClient' connection alive.
INFO:uamqp.c_uamqp:b'Failure: sending socket failed 10053.' (b'D:\a\1\s\src\vendor\azure-uamqp-c\deps\azure-c-shared-utility\adapters\socketio_win32.c':b'socketio_send':422)
INFO:uamqp.c_uamqp:b'xio_send failed' (b'D:\a\1\s\src\vendor\azure-uamqp-c\deps\azure-c-shared-utility\adapters\tlsio_schannel.c':b'send_chunk':484)
INFO:uamqp.c_uamqp:b'send_chunk failed' (b'D:\a\1\s\src\vendor\azure-uamqp-c\deps\azure-c-shared-utility\adapters\tlsio_schannel.c':b'internal_send':525)
INFO:uamqp.c_uamqp:b'send failed' (b'D:\a\1\s\src\vendor\azure-uamqp-c\deps\azure-c-shared-utility\adapters\tlsio_schannel.c':b'tlsio_schannel_send':1263)
INFO:uamqp.c_uamqp:b'xio_send failed' (b'D:\a\1\s\src\vendor\azure-uamqp-c\src\saslclientio.c':b'saslclientio_send_async':1190)
INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'D:\a\1\s\src\vendor\azure-uamqp-c\src\connection.c':b'on_bytes_encoded':268)

non deterministic error within uamqp 0.1.0 receiving messages

Unfortunately I am running into stability issues/errors when receiving messages with uamqp from a batch publish (more than 1 message at a time). This issue happens periodically on a batch send - it may happen on single message too (but haven't investigated this scenario).

I'd like to note I have not seen this happen with 0.1.0rc1 (most recent version I've integrated).

Here is the typical behavior I see. After batch sending the same payload a number of times (I just now repro'ed after 3 sends, a prior attempt I repro'ed after 2 sends) I see debug messages that look like:

event:
  annotations:
    iothub-connection-auth-generation-id: '636657185942037654'
    iothub-connection-auth-method: '{"scope":"hub","type":"sas","issuer":"iothub","acceptingIpFilterRule":null}'
    iothub-connection-device-id: d0
    iothub-enqueuedtime: 1531509971496
    iothub-message-source: Telemetry
    x-opt-enqueued-time: 1531509971636
    x-opt-offset: '86976'
    x-opt-sequence-number: 196
  origin: d0
  payload: Ping from Az CLI IoT Extension
  properties:
    application:
      a: '12345'
      b: somevalue
    system:
      correlation_id: '76232'
      message_id: '321'
 
uamqp.c_uamqp : Deallocating StringValue
uamqp.c_uamqp : Destroying StringValue
uamqp.c_uamqp : Deallocating StringValue
uamqp.c_uamqp : Destroying StringValue
uamqp.c_uamqp : Deallocating cMessage
uamqp.c_uamqp : Destorying cMessage
uamqp.c_uamqp : b'Value is not of type STRING' (b'C:\\Users\\antisch\\Documents\\GitHub\\azure-uamqp-python\\src\\vendor\\azure-uamqp-c\\src\\amqpvalue.c':b'amqpvalue_get_string':1112)
uamqp.c_uamqp : Destroying cMessageReceiver
uamqp.receiver : Message receiver b'receiver-link-705e64da-8aac-4151-862f-634094ea4ca3' state changed from MessageReceiverState.Open to MessageReceiverState.Closing
uamqp.c_uamqp : Destroying cLink
uamqp._async.client_async : Closing CBS session.
uamqp.c_uamqp : Destroying CBSTokenAuth
uamqp.c_uamqp : Destroying cSession
uamqp._async.client_async : Closing unshared connection.
uamqp.c_uamqp : Destroying Connection
uamqp.connection : Connection state changed from ConnectionState.OPENED to ConnectionState.END
uamqp.c_uamqp : Destroying SASLMechanism
uamqp.c_uamqp : Destroying XIO
uamqp.c_uamqp : Destroying XIO
uamqp : Deinitializing platform.
uamqp.c_uamqp : Deinitializing platform

It is a bit strange because the property values sent are always strings.

Not able to send message after using subject in MessageProperties

First of all I want to state that I am rather new to AMQP, so I may have misunderstood something here and if so, sorry...

I was expecting to be able to set the subject of a message to be sent, however when I try to send it I get a:

DEBUG:uamqp.client:Message error, retrying. Attempts: 1, Error: ClientMessageError('ErrorCodes.ClientError: expected bytes, uamqp.c_uamqp.StringValue found')

I see the same error if it try to use the get_properties_obj() function, which leads me to believe the following is a shortcut to reproduce my problem:

props = uamqp.message.MessageProperties(subject='test')
props.get_properties_obj()
Traceback (most recent call last):
File "", line 1, in
File "X:\env\lib\site-packages\uamqp\message.py", line 753, in get_properties_obj
self._set_attr('subject', properties)
File "X:\env\lib\site-packages\uamqp\message.py", line 742, in _set_attr
setattr(properties, attr, attr_value)
File ".\src/properties.pyx", line 140, in uamqp.c_uamqp.cProperties.subject.set
TypeError: expected bytes, uamqp.c_uamqp.StringValue found

If I instead set the message_id, everything seems to work properly.
I am currently running python 3.7 under windows with uamqp 1.0.0 installed by pip.

Let me end this post with a big thank you for providing this module. I am hoping to access an Azure servicebus using AMQP instead of the somewhat sluggish (in comparison) REST api.

Update Windows build support to VS2019

Right now the project can't build under VS2019, which is causing our Windows CI builds to have to stay on an older agent image. We should get this working under VS2019. Unfortunately just updating setuptools to >=34.4.0 and changing the CMake target is not enough, there are some platform problems that need to be resolved:

Python 3.5:
It's unable to locate a header file. My initial scan of this seems to indicate that as compared to the other Python version builds, the call to cl.exe is missing a number of include directories. This leads me to think that something in this version of Python (maybe distutils?) is too old to know how to locate those directories. We might have to figure out how to add them manually or whether there is just some component we can update with pip.

Python 3.6 x86:
This is odd because the x64 build passes. The x86 build always fails with a linker error, unresolved external symbol __except_handler4_common. I've tried using the -T host=x86 CMAKE flags to use the 32-bit compiler toolchain, the flag worked as documented but that didn't resolve the issue. At this point I'm a little stumped as to why this is happening and only for the x86 build.

simple_send to a different port

I'm trying to connect to an ActiveMQ broker which is running on the port 5673 instead of the standard 5671(because on 5671 we are running AMQP 0.91 with another broker).
I used the example:


import os
try:
    from urlparse import urlparse
except ImportError:
    from urllib.parse import urlparse

import uamqp
from uamqp import authentication

uri = "amqp://111.222.333.444:5673/examples/"
key_name = "user"
access_key = ******

def uamqp_send_simple():
    msg_content = b"Hello world"

    parsed_uri = urlparse(uri)
    plain_auth = authentication.SASLPlain(parsed_uri.hostname, key_name, access_key)

    uamqp.send_message(uri, msg_content, auth=plain_auth)
    print("Message sent!")

if __name__ == "__main__":
    uamqp_send_simple()

Anyway, it appears to be not working:

Traceback (most recent call last):
  File "test.py", line 30, in <module>
    uamqp_send_simple()
  File "test.py", line 26, in uamqp_send_simple
    uamqp.send_message(uri, msg_content, auth=plain_auth)
  File "/usr/local/lib/python2.7/dist-packages/uamqp/__init__.py", line 68, in send_message
    return send_client.send_all_messages()
  File "/usr/local/lib/python2.7/dist-packages/uamqp/client.py", line 775, in send_all_messages
    running = self.wait()
  File "/usr/local/lib/python2.7/dist-packages/uamqp/client.py", line 757, in wait
    running = self.do_work()
  File "/usr/local/lib/python2.7/dist-packages/uamqp/client.py", line 401, in do_work
    if not self.client_ready():
  File "/usr/local/lib/python2.7/dist-packages/uamqp/client.py", line 385, in client_ready
    if not self._client_ready():
  File "/usr/local/lib/python2.7/dist-packages/uamqp/client.py", line 531, in _client_ready
    self.message_handler.open()
  File "/usr/local/lib/python2.7/dist-packages/uamqp/sender.py", line 208, in open
    "Failed to open Message Sender. "
uamqp.errors.AMQPConnectionError: Failed to open Message Sender. Please confirm credentials and target URI.

Using wireshark I see that the TCP traffic is still going to 5671, so i suppose the port parameter is ignored at all.

Could you kindly post a working example on how to use uamqp on different TCP port.
The provided examples also they use environment variables, could be helpful to have a sample value of these environmental variables, i suppose is a standard form "amqps://host:port/channel".

Thanks

Improve Error Hierarchy

Give application level sdk a more clear structure.
Related issue: Azure/azure-sdk-for-python#5355

Receiver stops when concurrent receiver times out

Concurrent receivers sharing a connection stop receiving when one of the group times-out due to inactivity.
This is due to the underlying session being shutdown as the closed link endpoint is detected as an error.
Tracking the issue here:
Azure/azure-uamqp-c#228

Wrong timeout description

init.py, methods: receive_messages, receive_message you can find this text:

:param timeout: The timeout in seconds after which to return None if no messages
are retrieved. If set to 0 (the default), the receiver will not timeout and
will continue to wait for messages until interrupted.

But it's actualy in millisecond as written in the class ReceiveClient(AMQPClient) documentation from which are those methods created.

Async send stuck

I am using the client to send messages to the azure service bus. I noticed that some messages were never delivered.

The SendClientAsync.send_message_async method is stuck in the loop

while any([m for m in pending_batch if m.state not in constants.DONE_STATES]):    
    await self.do_work_async()

The message is in the state <MessageState.WaitingToBeSent: 0> and is not in self._pending_messages, so I am thinking that it might have never been added to the list due to some concurrency issue ?

Support Message grouping(sessionful)?

I'm using uamqp to connect with Azure Service Bus (enable session) and get following error:

uamqp.errors.LinkDetach: ErrorCodes.NotAllowed:
It is not possible for an entity that requires sessions to create a non-sessionful message receiver.

How can I receive the message with SessionId?

Thanks!

Improve idle timeout connection status transition

When connection idle timeout happens, the underlying C code would set the connection status to DISCARDING.

In previous implementation, the connection status would go through weird transition in the case of idle timeout:

OPENED->DISCARDING
DISCARDING->DISCARDING
DISCARDING->DISCARDING
...
DISCARDING->DISCARDING
DISCARDING->END
END->DISCARDING # This is the most weird part
Although the functionality is not affected, but the improper status transition is annoying.
This can be improved in state_changed of connection.py to handle the case of DISCARDING.

#131

Use AnyIO instead of AsyncIO

Is your feature request related to a problem? Please describe.
I'd like to use uamqp with trio. trio is another event loop implementation for Python which uses structured concurrency concepts.

Describe the solution you'd like
Instead of implementing asynchronous operations using asyncio the async_ops module should instead be implemented with AnyIO.
AnyIO abstracts the underlying event loop and allows you to choose which one you want to use.

Furthermore, it will save you from maintaining the asyncio implementation itself.
For example, currently the implementation uses the keyword argument loop when calling asyncio.shield. This will be deprecated in Python 3.10.

By using AnyIO, you will offload this maintenance burden.

Describe alternatives you've considered
I can already use this library with trio-asyncio but native support would be nicer.

Additional context
None.

Python 2.7 support (super() argument 1 must be type, not classobj)

Currently if installation is attempted on Python 2.7, the following error is given (this should be improved):

File "~\uamqp\setup.py", line 92, in __init__
    super(UAMQPExtension, self).__init__(name, sources=[ ])
TypeError: super() argument 1 must be type, not classobj

Support for ARM based releases (such as for Raspberry PI)

We have user requests for pre-built ARM based releases (like for armv7l) to support straight-forward uamqp usage on devices like Raspberry PI's.

Implement Event Processor Host pattern

Implement Event Processor Host pattern like in the previous Python SDK
https://github.com/Azure/azure-event-hubs-python/tree/develop/eventprocessorhost

Sending fails if an application property's value is a string of length 1

If one of the application properties' value is a string of length 1 (as an example, "t"), the application is unable to send (for both the async and sync senders).

Error message:

Message error, 3 retries exhausted (an integer is required)
uamqp.errors.MessageSendFailed: Failed to send message.

Code to reproduce: (slight modification of the repository samples)

application_properties={"test": "t"}
msg_content = b"hello world"

message = uamqp.Message(msg_content, application_properties=application_properties)

uri = "sb://{}/{}".format(live_eventhub_config['hostname'], live_eventhub_config['event_hub'])
sas_auth = authentication.SASTokenAuth.from_shared_access_key(
    uri, live_eventhub_config['key_name'], live_eventhub_config['access_key'])

target = "amqps://{}/{}".format(live_eventhub_config['hostname'], live_eventhub_config['event_hub'])
send_client = uamqp.SendClient(target, auth=sas_auth, debug=False)
send_client.send_message(message, close_on_done=True)

Re-Enable Python 3.8a in test matrix for azure-pipelines

The code is still present, commented. However, build failures against 3.8a are preventing passes. Given that this a pre-release python version, we do not want to block a release of uamqp on it. Investigation will happen the next release of the uamqp python library.

@annatisch FYI
Source issue

non deterministic - 'Error in xio_send.'

I have a service that is reading off the edge and writing to an eventhub for data ingest. The other day I got this unexpected error.


2019-07-01T09:14:37.674645159Z INFO:root:EventHubWriter Processing 300
2019-07-01T09:14:37.692856761Z INFO:uamqp.c_uamqp:b'Error in xio_send.' (b'/data/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c':b'write_outgoing_bytes':641)
2019-07-01T09:14:37.692998218Z INFO:uamqp.c_uamqp:b'Error in write_outgoing_bytes.' (b'/data/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/adapters/tlsio_openssl.c':b'tlsio_openssl_send':1374)
2019-07-01T09:14:37.693099347Z INFO:uamqp.c_uamqp:b'xio_send failed' (b'/data/src/vendor/azure-uamqp-c/src/saslclientio.c':b'saslclientio_send_async':1190)
2019-07-01T09:14:37.693196326Z INFO:uamqp.c_uamqp:b'Cannot send encoded bytes' (b'/data/src/vendor/azure-uamqp-c/src/connection.c':b'on_bytes_encoded':268)
2019-07-01T09:14:37.695223251Z INFO:uamqp.connection:Connection b'EHSender-9b940d7a-1bfd-450f-9b95-dd04ad872db5' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.END: 13>
2019-07-01T09:14:37.695336453Z INFO:uamqp.connection:Connection with ID b'EHSender-9b940d7a-1bfd-450f-9b95-dd04ad872db5' unexpectedly in an error state. Closing: False, Error: None
2019-07-01T09:14:37.695475969Z INFO:uamqp.c_uamqp:CBS error occured on connection b'EHSender-9b940d7a-1bfd-450f-9b95-dd04ad872db5'.
2019-07-01T09:14:37.695621513Z INFO:uamqp.sender:Message sender b'sender-link-eba09cf2-4d49-4d90-ac49-8d87697cbf3c' state changed from <MessageSenderState.Open: 2> to <MessageSenderState.Idle: 0> on connection: b'EHSender-9b940d7a-1bfd-450f-9b95-dd04ad872db5'
2019-07-01T09:14:37.696721740Z INFO:uamqp.c_uamqp:b'Connection not open' (b'/data/src/vendor/azure-uamqp-c/src/connection.c':b'connection_encode_frame':2048)
2019-07-01T09:14:37.696836138Z INFO:uamqp.c_uamqp:b'Failed session send transfer' (b'/data/src/vendor/azure-uamqp-c/src/link.c':b'link_transfer_async':1465)
2019-07-01T09:14:37.696931524Z INFO:uamqp.c_uamqp:b'Invalid argument (list=(nil), item=0x5575ccee7a90)' (b'/data/src/vendor/azure-uamqp-c/deps/azure-c-shared-utility/src/singlylinkedlist.c':b'singlylinkedlist_remove':108)
2019-07-01T09:14:37.697026817Z INFO:uamqp.c_uamqp:b'Error removing pending delivery from the list' (b'/data/src/vendor/azure-uamqp-c/src/link.c':b'link_transfer_async':1468)
2019-07-01T09:14:37.697121349Z INFO:uamqp.c_uamqp:b'Error in link transfer' (b'/data/src/vendor/azure-uamqp-c/src/message_sender.c':b'send_one_message':544)
2019-07-01T09:14:37.697227728Z INFO:uamqp.c_uamqp:b'Error sending message' (b'/data/src/vendor/azure-uamqp-c/src/message_sender.c':b'messagesender_send_async':916)
2019-07-01T09:14:38.361936367Z /home/startSync.sh: line 30:     7 Segmentation fault      (core dumped) python -u update.py

The first message is from my eventhub writer.

The code snippet that produced this error is:

        logging.info("EventHubWriter Processing " + str(len(new_records)))
        start = timer()

        chunks = self.get_batches(new_records, 250)

        try:
            for chunk in chunks:
                ticks = self.get_batch_ticks(job, chunk)
                if ticks:
                    data = EventData(batch=ticks)
                    self.sender.send(data)

        except Exception as e:
            logging.warning("Event Hub write: (%s)", e)
            raise e

        finally:
            end = timer()
            logging.info("\tEventHubWriter Processing complete: " + str(end - start) + " seconds")
            # client.stop()

What was most surprising to me was an exception was never raised up to the app level.

My process is checkpointed. On restart of the system the same batch was reprocessed without incident.

I'm curious what may have caused the issue and why my code didn't receive an exception, and what, if anything, I could/should be doing differently.

I suspect this may be related to issue 52, but it wasn't clear if they are the same or not.

Scheduled enqueue using annotations on Azure SB

Hey, Im trying to schedule a message enqueue on service bus, here is the sample code:

with topic_client.get_sender() as sender:
        message = Message(b"sample topic message")
        message.annotations = {}
        TWO_HOURS = 2 * 60 * 60
        delay_time = int((time.time() + TWO_HOURS) * 1000)
        message.annotations[types.AMQPSymbol(b"x-opt-scheduled-enqueue-time")] = delay_time
        sender.send(message)

Theoretically, it should work, but in ServiceBus Explorer the line that show "ScheduledEnqueueTimeUTC" is empty. Am I doing something wrong? Thanks!

ImportError: cannot import name 'c_uamqp'

I am trying to run python script(3.6) as WebJob in App Service Plan to write and read service bus queue messages. In Local machine, It working fine but during hosting on Cloud, It throwing error

[06/11/2019 17:11:06 > 34a114: ERR ] File "PythonApplication1.py", line 2, in
[06/11/2019 17:11:06 > 34a114: ERR ] from azure.servicebus.control_client import ServiceBusService
[06/11/2019 17:11:06 > 34a114: ERR ] File "D:\local\Temp\jobs\continuous\pythontestjob\vxumnp54.5g3\azure\servicebus_init_.py", line 10, in
[06/11/2019 17:11:06 > 34a114: ERR ] from azure.servicebus.common.message import Message, BatchMessage, PeekMessage, DeferredMessage
[06/11/2019 17:11:06 > 34a114: ERR ] File "D:\local\Temp\jobs\continuous\pythontestjob\vxumnp54.5g3\azure\servicebus\common\message.py", line 10, in
[06/11/2019 17:11:06 > 34a114: ERR ] import uamqp
[06/11/2019 17:11:06 > 34a114: ERR ] File "D:\local\Temp\jobs\continuous\pythontestjob\vxumnp54.5g3\uamqp_init_.py", line 10, in
[06/11/2019 17:11:06 > 34a114: ERR ] from uamqp import c_uamqp # pylint: disable=import-self
[06/11/2019 17:11:06 > 34a114: ERR ] ImportError: cannot import name 'c_uamqp'
[06/11/2019 17:11:06 > 34a114: SYS ERR ] Job failed due to exit code 1

Close client when the operation is timeout

In receive_message_batch, when the operation is timeout, we simply set the self._shutdown to be True and don't close/reset ReceiveClient.

In this situation, receive_message_batch can still be called but the do_work function will simply returns as it checks that self._shutdown is True.

This is related to the RP: #90

X.509 Client Authentication

I have been trying (unsuccessfully) to build a client that uses TLS client authentication via X.509 certificates. After digging around the code I think that right now only server authentication via X.509 is possible and client authentication is restricted to SASL. Is that the case or am I missing something?

Could this feature be considered for a future release?

Thanks!

deallocation issues with async connection + client

I have experienced this issue with the latest uamqp v0.2.1 and below (but earlier versions may have manifested slightly differently).

Every so often I run into one of two issues when trying to gracefully close out uamqp receivers + connection. This happens specifically with async components.

The first is a deadlock (freeze behavior) in the process when trying to shut down a connection (in this case a shared connection).

The debug logs look like this:

uamqp.c_uamqp : Destroying CompositeValue
uamqp.c_uamqp : Deallocating CompositeValue
uamqp.c_uamqp : Destroying CompositeValue
uamqp.c_uamqp : Deallocating cLink
uamqp.c_uamqp : Deallocating cMessageReceiver
uamqp._async.client_async : CBS session pending.
uamqp._async.client_async : Shared connection remaining open.
uamqp.c_uamqp : Deallocating cSource
uamqp.c_uamqp : Destroying cSource
uamqp.c_uamqp : Deallocating StringValue
uamqp.c_uamqp : Destroying StringValue
uamqp.c_uamqp : Deallocating StringValue
uamqp.c_uamqp : Destroying StringValue
uamqp.c_uamqp : Deallocating SymbolValue
uamqp.c_uamqp : Destroying SymbolValue
uamqp._async.connection_async : Shutting down connection.
uamqp.authentication.cbs_auth_async : Shutting down CBS session.

End

In this state, after the last shutting down CBS session message you have to externally kill the process to recover.

The second issue is that a number of 'Error in sys.excepthook:' errors will appear and the returned status code is 255.

The logs look like this:

uamqp.c_uamqp : Deallocating cSource
uamqp.c_uamqp : Destroying cSource
uamqp.c_uamqp : Deallocating StringValue
uamqp.c_uamqp : Destroying StringValue
uamqp.c_uamqp : Deallocating XIO
uamqp.c_uamqp : Deallocating SASLMechanism
uamqp.c_uamqp : Deallocating XIO
uamqp.c_uamqp : Deallocating CompositeValue
uamqp.c_uamqp : Destroying CompositeValue
uamqp.c_uamqp : Deallocating CompositeValue
uamqp.c_uamqp : Destroying CompositeValue
uamqp.c_uamqp : Deallocating Connection
uamqp.c_uamqp : Deallocating cSession
uamqp.c_uamqp : Deallocating cLink
uamqp.c_uamqp : Deallocating cMessageReceiver
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook: