Git Product home page Git Product logo

Comments (15)

rflamary avatar rflamary commented on September 12, 2024 1

Hello,

This problem is a known bug from pypi discussed here
pypa/pip#2478

and here
pypa/setuptools#391

Basically in order to install POT you need to have cython and numpy already installed because they are dependency at build time. From what i saw this is a common problem. I think it could be addressed if we provided compiled eggs (since they won't require building anymore) but its a lot of work.

a working Dockerfile below

FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install python3.7 python3-pip build-essential git
RUN pip3 install cython numpy
RUN pip3 install POT

I agree it's not optimal but i don't think we can do anything until the question is handled by pypi

from pot.

rflamary avatar rflamary commented on September 12, 2024

Hello @Adoni and thank you for the bug report.

This is very surprising since I cannot seem to reproduce the bug. On what platform are you? When using pip, the variable install_requires defined in the setup.py should install automatically cython when building POT.

from pot.

Adoni avatar Adoni commented on September 12, 2024

It's mac os in my computer and I'm using anaconda.

However, my friend reproduced this bug in Linux, who didn't use Anaconda. So to some degree I think this is a common problem.

from pot.

rflamary avatar rflamary commented on September 12, 2024

Hello i'm sorry but I don't have enough information to reproduce the bug.

On anaconda we recommend to use conda forge that take care of all compilation.

We now have an issue template with the list of required information :
https://github.com/rflamary/POT/blob/master/.github/ISSUE_TEMPLATE/bug_report.md

from pot.

leotrs avatar leotrs commented on September 12, 2024

I have been able to reproduce the bug by installing POT from a requirements.txt file. Example bash session follows.

leo@lily: test-pot$ mkvirtualenv test-pot
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/leo/.virtualenvs/test-pot/bin/python2
Also creating executable in /home/leo/.virtualenvs/test-pot/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/preactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/postactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/get_env_details

(test-pot) leo@lily: test-pot$ echo "matplotlib==2.0.2
> numpy==1.13.1
> networkx==1.11
> scikit-learn==0.18.2
> scipy==0.19.1
> pandas==0.19.1
> POT==0.5.1" > requirements.txt

(test-pot) leo@lily: test-pot$ cat requirements.txt 
matplotlib==2.0.2
numpy==1.13.1
networkx==1.11
scikit-learn==0.18.2
scipy==0.19.1
pandas==0.19.1
POT==0.5.1

(test-pot) leo@lily: test-pot$ pip install -r requirements.txt 
<suppressed output...>
Collecting POT==0.5.1 (from -r requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/28/4b/7aaa1f840a359f5953dd378e0237fa8faf9b0a415ff7282b7375fbe68d27/POT-0.5.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-zvHRKm/POT/setup.py", line 7, in <module>
        from Cython.Build import cythonize
    ImportError: No module named Cython.Build
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zvHRKm/POT/

from pot.

ncourty avatar ncourty commented on September 12, 2024

Hi @leotrs, are you working on OSX Mojave ? It seems the problem comes from a bad include directory with the compiler used by Cython. See this issue #71 (comment) with the patch that was supposed to work at that time...

from pot.

ncourty avatar ncourty commented on September 12, 2024

wait I might have reacted too quickly.... Is cython installed on your side ? Maybe we just forgot to add it in the requirements as it usually ships with most python distrib

from pot.

leotrs avatar leotrs commented on September 12, 2024

No. I'm on Ubuntu 16.04. I should clarify that I was able to install POT by (i) removing it from the requirements.txt file, (ii) installing the remaining requirements using pip install -r requirements.txt, and (iii) installing POT on its own pip install pot.

from pot.

leotrs avatar leotrs commented on September 12, 2024

I believe it must be part of the requirements, since doing just pip install pot works for me. The only time it doesn't work is when it is being installed from a requirements.txt file.

from pot.

leotrs avatar leotrs commented on September 12, 2024

Any updates on this?

from pot.

rflamary avatar rflamary commented on September 12, 2024

Hello @leotrs,

Sorry for the late reply but we have been quite busy. I have seen the discussion on the bug you refered and I can tell you the POT is actively maintained. e have peaks of activity followed by limited availability but we try to handle bugs in a timely manner (except this one apparently ;) )

This is a weird bug and I am trying to reproduce your bug but i'm on ubuntu 18.04.

This is what i have

[:~/test2] master ± virtualenv test-pot
Using base prefix '/usr'
New python executable in /home/rflamary/test2/test-pot/bin/python3
Also creating executable in /home/rflamary/test2/test-pot/bin/python
Installing setuptools, pip, wheel...done.
[:~/test2] master ± source test-pot/bin/activate
[:~/test2] [test-pot] master ± echo "matplotlib==2.0.2
> > numpy==1.13.1
> > networkx==1.11
> > scikit-learn==0.18.2
> > scipy==0.19.1
> > pandas==0.19.1
> > POT==0.5.1" > requirements.txt
[:~/test2] [test-pot] master ±  cat requirements.txt 
matplotlib==2.0.2
> numpy==1.13.1
> networkx==1.11
> scikit-learn==0.18.2
> scipy==0.19.1
> pandas==0.19.1
> POT==0.5.1
[:~/test2] [test-pot] master ± cat requirements.txt 
matplotlib==2.0.2
numpy==1.13.1
networkx==1.11
scikit-learn==0.18.2
scipy==0.19.1
pandas==0.19.1
POT==0.5.1
[:~/test2] [test-pot] master ±  pip install -r requirements.txt 
Collecting matplotlib==2.0.2 (from -r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/60/d4/6b6d8a7a6bc69a1602ab372f6fc6e88ef88a8a96398a1a25edbac636295b/matplotlib-2.0.2-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy==1.13.1 (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/59/e2/57c1a6af4ff0ac095dd68b12bf07771813dbf401faf1b97f5fc0cb963647/numpy-1.13.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting networkx==1.11 (from -r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/d3/2c/e473e54afc9fae58dfa97066ef6709a7e35a1dd1c28c5a3842989322be00/networkx-1.11-py2.py3-none-any.whl
Collecting scikit-learn==0.18.2 (from -r requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/d1/51/ca5be39c576c981cf8b8359fcb1ee49ca43d59b833415205cec5ef5c0fcd/scikit_learn-0.18.2-cp36-cp36m-manylinux1_x86_64.whl
Collecting scipy==0.19.1 (from -r requirements.txt (line 5))
  Using cached https://files.pythonhosted.org/packages/0e/46/da8d7166102d29695330f7c0b912955498542988542c0d2ae3ea0389c68d/scipy-0.19.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting pandas==0.19.1 (from -r requirements.txt (line 6))
Collecting POT==0.5.1 (from -r requirements.txt (line 7))
Collecting cycler>=0.10 (from matplotlib==2.0.2->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib==2.0.2->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl
Collecting python-dateutil (from matplotlib==2.0.2->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.0,!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 (from matplotlib==2.0.2->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/de/0a/001be530836743d8be6c2d85069f46fecf84ac6c18c7f5fb8125ee11d854/pyparsing-2.3.1-py2.py3-none-any.whl
Collecting six>=1.10 (from matplotlib==2.0.2->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting decorator>=3.4.0 (from networkx==1.11->-r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/f1/cd/7c8240007e9716b14679bc217a1baefa4432aa30394f7e2ec40a52b1a708/decorator-4.3.2-py2.py3-none-any.whl
Collecting cython (from POT==0.5.1->-r requirements.txt (line 7))
  Using cached https://files.pythonhosted.org/packages/e1/fd/711507fa396064bf716493861d6955af45369d2c470548e34af20b79d4d4/Cython-0.29.6-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: six, cycler, pytz, python-dateutil, pyparsing, numpy, matplotlib, decorator, networkx, scikit-learn, scipy, pandas, cython, POT
Successfully installed POT-0.5.1 cycler-0.10.0 cython-0.29.6 decorator-4.3.2 matplotlib-2.0.2 networkx-1.11 numpy-1.13.1 pandas-0.19.1 pyparsing-2.3.1 python-dateutil-2.8.0 pytz-2018.9 scikit-learn-0.18.2 scipy-0.19.1 six-1.12.0

And it automatically installed cython which seemed to be the bug in what you sent us. I tried it with python3.6 and python2.7 and it worked.

I'm not familiar with virtualenvwrapper but i don't this the discrepancy come from here. So it might be a problem of version from pip? The install_requires parameter has only been added recently and if you have an old pip he might discard it and not install cython that is required to compile POT. if you want to be sure to compile whatever the pip version you can always add cython in the requirement (I know nobody wants that but we do not provide pre-compiled eggs on pipy).

Edit: reran the test after cleaning PYTHONPATH variable.

from pot.

leotrs avatar leotrs commented on September 12, 2024

@rflamary thank you for looking into this. I don't think it's the version of pip since we are replicating the bug across a number of computers.

from pot.

rflamary avatar rflamary commented on September 12, 2024

OK I managed to reproduce the problem.
This occurs on python2.7 when i force it for the virtual environment

virtualenv --python=/usr/bin/python2.7 test_pot_py27

It works on python 3.6...

I'm wondering if it is a problem related to pypa/pip#2478
Its the same pip version for both, this is a weird bug. install_requires should work on both 2.7 and 3.6 but it seems to fail installing Cython before POT on 2.7...

edit: wrong link

from pot.

joshua-gould avatar joshua-gould commented on September 12, 2024

Here's a Dockerfile that shows the package failing using pip install with the error File "/tmp/pip-build-y26thcux/POT/setup.py", line 7, in
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'

FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install python3.7 python3-pip build-essential
RUN pip3 install POT

from pot.

rflamary avatar rflamary commented on September 12, 2024

This is perfect for debug thank you.

I will have a look (probably after the NeurIPS deadline) but i think it might be corrected by properly defining build dependencies as discussed https://packaging.python.org/specifications/declaring-build-dependencies/ and https://www.python.org/dev/peps/pep-0518/.

from pot.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.