ARM (Raspberry Pi 4) error on import: undefined symbol: __atomic_fetch_sub_8 about pynng HOT 8 CLOSED

I was wondering how a proper solution would look like, adding a platform check in build_pynng.py or some cmake magic like in the nng library?

That's a great question and I'm open to input. For now I added a change to link libatomic... in a pretty hacky way, and better fixes are welcome. (The change I made is ac4d0a6 if anyone wants to check it out)

The reason this happens is that I am building the upstream libnng as a static library, which essentially throws away all the information that Garrett's carefully crafted CMakeLists.txt file has about what libraries need to be linked. The reason I build and link it statically instead of a shared library is that it makes vendoring the .whl files really easy; no external dependencies, and everything is just vendored right in there.

It turns out it is possible to include shared libs in .whl files, at least on Linux, but I think for now it's just easier to track what libraries need to be linked manually instead of trying to vendor.

I'm closing for now because the change I made just now should fix this.

from pynng.

Hi @codypiersall,

Thanks ever so much for the quick response and explanation. That all makes sense, although I'm a C++ dev by trade and fairly new to Python so haven't played with LD_PRELOAD before now. Hadn't spotted that it was actually a command line with that declaration BEFORE the "python" program call and I could just add my command line parameters on the end. Linux is strange and I've had a long day :)

Thanks, I'm up and running now.

Thanks for the awesome library!

from pynng.

Hi @matthiasharrer, Thanks for opening this issue.

I believe that the latest nng should fix this change, so it's unfortunate that it didn't fix it for you.

I have worked around this on a Pi 4 by setting LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1. I should have opened an issue when I ran into this problem!

(I believe this happens because of symbols moving out of libc and into libatomic, FWIW.)

from pynng.

Hi @codypiersall thanks a lot for your response.

Do you think it should be enough to change the nng ref like this:
matthiasharrer@788d04a

And then install the package with pip install --no-cache-dir git+https://github.com/matthiasharrer/pynng.git@788d04a60e32d9424448bf7a8aeca18077a77471 or am I missing something here?

from pynng.

Hi @codypiersall,

I've been trying to get this working on the RPi Zero W. Same issue described above. Installing via PIP works fine, but then I hit this error at runtime in my python scripts. Basically from the putty console I'm running my file using "python main.py ...etc" which means its compiled and run there and then - but fails the compile.

Any idea how to implement this workaround under these circumstances please? Is there a global LD_PRELOAD environment variable that can be changed in python 3?

Or better, is there a change that is required to the pip package to allow pynng to work out of the box?

FYI this is the error:

file "/usr/local/lib/python3.7/dist-packages/pynng/init.py", line 3, in
from ._nng import lib, ffi
ImportError: /usr/local/lib/python3.7/dist-packages/pynng/_nng.abi3.so: undefined symbol: __atomic_fetch_sub_8

Many thanks

from pynng.

Hi @cpresty,

I think you can already solve your issue by starting Python this way on your Rpi:

LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1 python main.py ...etc.

Is there a global LD_PRELOAD environment variable that can be changed in python 3?

LD_PRELOAD is the environment variable that needs to be changed. But, by the time Python is running, it would be too late; you have to set the variable before python is running. So changing it in Python wouldn't work; LD_PRELOAD won't do anything at that point.

LD_PRELOAD tells the linker to load libraries before starting a program. In general LD_PRELOAD is used for all sorts of different reasons; here, we're using it because on the Raspberry Pi, pynng requires functions that don't exist in any of the libraries it gets linked against, so we tell the linker to go ahead and link those.

is there a change that is required to the pip package to allow pynng to work out of the box?

I think pynng's setup.py script could be changed to make it work, and then it will Just Work™ when installed via pip.

from pynng.

This is still an issue on all platforms which need to link libatomic (armv7, armv5, mips etc). Also the prebuild wheel file from PyPI does not work. I was able to work around by adding atomic to the list of libraries in build_pynng.py.

I was wondering how a proper solution would look like, adding a platform check in build_pynng.py or some cmake magic like in the nng library?

from pynng.

Same here, failed to build openvino by this error message...

from pynng.