import shenfun: Illegal instruction (core dumped) error about shenfun HOT 18 CLOSED

Thank you!

I needed to fix a typo above, comilers -> compilers and add numba to the list but I can say that shenfun now imports on the server!

I'm now going to test some demos but I can say that after adding matplotlib, RayleighBenardRK3.py successfully runs.

Thanks again Mikael, the help is greatly appreciated.

from shenfun.

Bugs are rarely easy beats to tame and this one sounds especially challenging. Congratulations on finding it and I really appreciate you fixing it, and many others should too. I will update shenfun and make sure everything is good on my end.

from shenfun.

Hi

Thanks for reporting the issue, but this is not reproducible for my linux laptop:

$ conda create --name shenfuntest python=3.6.0 shenfun mpich nomkl
$ conda activate shenfuntest  
$ python
Python 3.6.0 | packaged by conda-forge | (default, Feb  9 2017, 14:36:55) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shenfun
>>> 

For future reference, you need to provide at least something that shows where and how things go wrong. There is nothing to go by in just a segfault. Could be anything. This error is probably not an error in shenfun. For example, can you import mpi4py, numpy?

from shenfun.

Yes, I could able to import these two packages, snapshot of the same is pasted below for your information.
Let me know what other things could impact such failure.

$ conda activate shenfuntest
(shenfuntest) $ python
Python 3.6.0 | packaged by conda-forge | (default, Feb 9 2017, 14:36:55)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.

import numpy
import mpi4py
import shenfun
Illegal instruction
(shenfuntest) $

from shenfun.

I am trying to install shenfun on a fairly larger server and have installed Anaconda.

The good news is that when I installed mpi4py-fft it worked beautifully.

The bad news is that when I tried and set up shenfun, I get an illegal instruction error when I import shenfun. Using faulthandler, I found the following infomration that I can't really make heads or tails out of. Any idea what might be wrong?

`$ python -q -X faulthandler
>>> import shenfun
Fatal Python error: Illegal instruction

Current thread 0x00002ba78b6fcec0 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1109 in exec_module
  File "<frozen importlib._bootstrap>", line 671 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load

File"/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.8/sitepackages/shenfun/optimization/cython/init.py", line 1 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.8/sitepackages/shenfun/chebyshev/la.py", line 6 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 219 in _call_with_frames_removed
File "", line 1042 in _handle_fromlist
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.8/sitepackages/shenfun/chebyshev/init.py", line 2 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 219 in _call_with_frames_removed
File "", line 1042 in _handle_fromlist
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.8/sitepackages/shenfun/init.py", line 37 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 1 in
Illegal instruction
`

from shenfun.

No idea, except that it's cython related. Is this installed from conda-forge? Perhaps you need to compile it by hand on the server?

from shenfun.

Thanks for pointing that out, now it seems pretty obvious that cython was the initial source of the problem.

Yes, I did use conda-forge and noticed that cython was not installed.

I pip installed cython but I fear that might have used a default version of cython that is on the server. I will try again with a new install and include cython on the list.

By default, the server is openmpi. Since shenfun needs mpich, I presume I should remove all openmpi libraries and load mpich, before I do the conda create? Or is conda smart enough to do all of that?

from shenfun.

Did you check that you used anaconda's pip? It's a common mistake to use system pip.

Openmpi should not be a problem. I had some issues with them in the past, but I don't remember why. You should probably use the mpi that has been installed on the server, assuming the people that installed them used systemspecific optimizations. Conda-forge versions do not know about you system:-)

from shenfun.

I believe that I did use the conda version of pip but I removed all that I had before and did a new conda create now with cython, just to be sure.

I also used python 3.7.6, which is what I have on my local machine, and that helped.

Good news: I can import cython and the errors I had previously go away.

Bad news: there is a new error, which I get when I install mpi4py_fft.

`$ python -q -X faulthandler
>>> import cython
>>> import mpi4py
>>> import mpi4py_fft

Traceback (most recent call last):
File "", line 1, in
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.7/site-packages/mpi4py_fft/init.py", line 22, in
from .distarray import DistArray, newDistArray, Function
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.7/site-packages/mpi4py_fft/distarray.py", line 4, in
from mpi4py import MPI
ImportError: libhcoll.so.1: cannot open shared object file: No such file or directory
>>> import shenfun
Traceback (most recent call last):
File "", line 1, in
File "/home/fpoulin/software/software/anaconda3/envs/shenfun/lib/python3.7/site-packages/shenfun/init.py", line 36, in
from mpi4py import MPI
ImportError: libhcoll.so.1: cannot open shared object file: No such file or directory
`

from shenfun.

Minor update:

I can confirm that when I install mpi4py-fft with python 3.8.2 it just works. However, when I use python 3.7.6, it doesn't seem to work on the server. I have another computer that has them both installed with 3.8.2 so I will try that again, and see if I have any more success.

from shenfun.

Minor update:

When I install shenfun using python 3.8.2 I get that mpi4py_fft works, but then I get the fatal python error: illegal instruction, when I try and run shenfun. Since this conda install worked on one of my computers I'm not sure why it doesn't work with 3.8.2 on the server. Hmmm....

from shenfun.

Sorry to bother you with this again but any idea why I might get a failure of shenfun when I use this particular version of python?

from shenfun.

Sorry, no idea. Shenfun is tested automatically on python 3.6, 3.7 and 3.8, so it's probably not a regular python thing.

from shenfun.

Thaks. It turns out that when I tried the exact line on my laptop I ended up getting the same error.

This leads me to believe that I should try and find out exactly what I did to create.

Unfortunately, when I try and create shenfun, as I did before, it doesn't seem to work.

conda create --name sf5 -c conda-forge shenfun mpich python=3.8.2

This yields similar errors as far as I can tell.

$ python -q -X faulthandler

import shenfun
Fatal Python error: Illegal instruction

Current thread 0x00007fa629660740 (most recent call first):
File "", line 219 in _call_with_frames_removed
File "", line 1109 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "/home/fpoulin/software/anaconda3/envs/sf5/lib/python3.8/site-packages/shenfun/optimization/cython/init.py", line 1 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "/home/fpoulin/software/anaconda3/envs/sf5/lib/python3.8/site-packages/shenfun/chebyshev/la.py", line 6 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 219 in _call_with_frames_removed
File "", line 1042 in _handle_fromlist
File "/home/fpoulin/software/anaconda3/envs/sf5/lib/python3.8/site-packages/shenfun/chebyshev/init.py", line 2 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 219 in _call_with_frames_removed
File "", line 1042 in _handle_fromlist
File "/home/fpoulin/software/anaconda3/envs/sf5/lib/python3.8/site-packages/shenfun/init.py", line 37 in
File "", line 219 in _call_with_frames_removed
File "", line 783 in exec_module
File "", line 671 in _load_unlocked
File "", line 975 in _find_and_load_unlocked
File "", line 991 in _find_and_load
File "", line 1 in
Illegal instruction (core dumped)

from shenfun.

conda deactivate
conda create --name sf5 -c conda-forge shenfun mpich python=3.8.2
conda activate sf5
export PYTHONPATH=''
python -c 'import shenfun'

Works for me on my mac. I can try in a docker linux image later.

from shenfun.

I have tried this on two linux machine and am getting the same problem so if you could check, sometime, that would be greatly appreciated.

If i wanted to check to see how my working version of shenfun was installed, is there an easy way to find that out? I should probably try that again as a sanity check.

from shenfun.

So I tried it in a docker image and I can reproduce the illegal instruction. Solution for me in the docker container, within sf5 environment:

conda install -c conda-forge comilers cython
git clone https://github.com/spectralDNS/shenfun.git
cd shenfun
python setup.py build_ext -i
export PYTHONPATH=~/shenfun
python -c 'import shenfun' 

runs without mistake. So there must be an incompatibility between the shenfun built by conda-forge some of the packages you get when installing with python=3.8.2. I don't know what it is, but the solution is to build it locally.

from shenfun.

@francispoulin I just found a bug that I think was causing this segfault! There's a line in setup.py that adds compiler options, like -O3 etc. For my native computer I had included -march=native. But I just found that as an unintended side-effect -march=native was also added to the conda-forge build! The bug has probably hit a lot more people than just you, so it's really annoying that it has been there for so long and I didn't figure it out before now.

from shenfun.