[QST] Importing cudf in Jupyter notebook gives error about cudf HOT 6 OPEN

Thanks for the question. Can you print sys.executable and sys.path and make sure your notebook is using the right environment? Sometimes notebooks use a different kernel than the environment you would expect when launching. Also please share how you installed cudf (conda? pip?) and version/platform information.

from cudf.

I am running Ubuntu 20.04.6 LTS

cudf is installed by:
conda create -n rapids-24.04 -c rapidsai -c conda-forge -c nvidia rapids=24.04 python=3.10 cuda-version=11.8

From command line:

(rapids-24.04) $ python
>>> print(f'sys.executable={sys.executable}')
sys.executable=/home/recsys-api/mnt/miniconda3/envs/rapids-24.04/bin/python3
>>> print(f'sys.path={sys.path}')
sys.path=['', '/home/recsys-api/mnt/miniconda3/envs/rapids-24.04/lib/python310.zip', '/home/recsys-api/mnt/miniconda3/envs/rapids-24.04/lib/python3.10', '/home/recsys-api/mnt/miniconda3/envs/rapids-24.04/lib/python3.10/lib-dynload', '/home/recsys-api/.local/lib/python3.10/site-packages', '/home/recsys-api/mnt/miniconda3/envs/rapids-24.04/lib/python3.10/site-packages']

Jupyter server is started in rapids-24.04, in the notebook:

sys.executable=/usr/bin/python3

sys.path=['/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '', '/home/recsys-api/.local/lib/python3.10/site-packages', '/usr/local/lib/python3.10/dist-packages', '/usr/lib/python3/dist-packages']

Jupyterlab version is 4.1.8

Thanks!

from cudf.

Hi @Ullar-Kask ,
It seems like you might have some environment issues stemming from the cuDF installation and your jupyter notebook server not being co-located. One thing that might help if you want to use a jupyter notebook with cuDF is to try creating the rapids conda environment with the notebook package included from the start. This is your installation command with notebook appended to the end:

conda create -n rapids-24.04 -c rapidsai -c conda-forge -c nvidia rapids=24.04 python=3.10 cuda-version=11.8 notebook

This should avoid having to separately install notebook which I could see maybe leading to issues like what you're seeing. With this command I'm able to start a notebook from within the rapids-24.04 environment and import cudf successfully. Can you try it this way and see if that works for you?

from cudf.

sys.executable=/usr/bin/python3 indicates that your Jupyter kernel is not using the RAPIDS conda environment you created.

You might need something like this? https://stackoverflow.com/questions/39604271/conda-environments-not-showing-up-in-jupyter-notebook

I think that installing jupyter into the RAPIDS conda environment and using that may work as well.

from cudf.