Comments (15)
Hello,
This problem is a known bug from pypi discussed here
pypa/pip#2478
and here
pypa/setuptools#391
Basically in order to install POT you need to have cython and numpy already installed because they are dependency at build time. From what i saw this is a common problem. I think it could be addressed if we provided compiled eggs (since they won't require building anymore) but its a lot of work.
a working Dockerfile below
FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install python3.7 python3-pip build-essential git
RUN pip3 install cython numpy
RUN pip3 install POT
I agree it's not optimal but i don't think we can do anything until the question is handled by pypi
from pot.
Hello @Adoni and thank you for the bug report.
This is very surprising since I cannot seem to reproduce the bug. On what platform are you? When using pip, the variable install_requires
defined in the setup.py
should install automatically cython when building POT.
from pot.
It's mac os in my computer and I'm using anaconda.
However, my friend reproduced this bug in Linux, who didn't use Anaconda. So to some degree I think this is a common problem.
from pot.
Hello i'm sorry but I don't have enough information to reproduce the bug.
On anaconda we recommend to use conda forge that take care of all compilation.
We now have an issue template with the list of required information :
https://github.com/rflamary/POT/blob/master/.github/ISSUE_TEMPLATE/bug_report.md
from pot.
I have been able to reproduce the bug by installing POT from a requirements.txt
file. Example bash session follows.
leo@lily: test-pot$ mkvirtualenv test-pot
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/leo/.virtualenvs/test-pot/bin/python2
Also creating executable in /home/leo/.virtualenvs/test-pot/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/preactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/postactivate
virtualenvwrapper.user_scripts creating /home/leo/.virtualenvs/test-pot/bin/get_env_details
(test-pot) leo@lily: test-pot$ echo "matplotlib==2.0.2
> numpy==1.13.1
> networkx==1.11
> scikit-learn==0.18.2
> scipy==0.19.1
> pandas==0.19.1
> POT==0.5.1" > requirements.txt
(test-pot) leo@lily: test-pot$ cat requirements.txt
matplotlib==2.0.2
numpy==1.13.1
networkx==1.11
scikit-learn==0.18.2
scipy==0.19.1
pandas==0.19.1
POT==0.5.1
(test-pot) leo@lily: test-pot$ pip install -r requirements.txt
<suppressed output...>
Collecting POT==0.5.1 (from -r requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/28/4b/7aaa1f840a359f5953dd378e0237fa8faf9b0a415ff7282b7375fbe68d27/POT-0.5.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-zvHRKm/POT/setup.py", line 7, in <module>
from Cython.Build import cythonize
ImportError: No module named Cython.Build
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-zvHRKm/POT/
from pot.
Hi @leotrs, are you working on OSX Mojave ? It seems the problem comes from a bad include directory with the compiler used by Cython. See this issue #71 (comment) with the patch that was supposed to work at that time...
from pot.
wait I might have reacted too quickly.... Is cython installed on your side ? Maybe we just forgot to add it in the requirements as it usually ships with most python distrib
from pot.
No. I'm on Ubuntu 16.04. I should clarify that I was able to install POT by (i) removing it from the requirements.txt
file, (ii) installing the remaining requirements using pip install -r requirements.txt
, and (iii) installing POT on its own pip install pot
.
from pot.
I believe it must be part of the requirements, since doing just pip install pot
works for me. The only time it doesn't work is when it is being installed from a requirements.txt
file.
from pot.
Any updates on this?
from pot.
Hello @leotrs,
Sorry for the late reply but we have been quite busy. I have seen the discussion on the bug you refered and I can tell you the POT is actively maintained. e have peaks of activity followed by limited availability but we try to handle bugs in a timely manner (except this one apparently ;) )
This is a weird bug and I am trying to reproduce your bug but i'm on ubuntu 18.04.
This is what i have
[:~/test2] master ± virtualenv test-pot
Using base prefix '/usr'
New python executable in /home/rflamary/test2/test-pot/bin/python3
Also creating executable in /home/rflamary/test2/test-pot/bin/python
Installing setuptools, pip, wheel...done.
[:~/test2] master ± source test-pot/bin/activate
[:~/test2] [test-pot] master ± echo "matplotlib==2.0.2
> > numpy==1.13.1
> > networkx==1.11
> > scikit-learn==0.18.2
> > scipy==0.19.1
> > pandas==0.19.1
> > POT==0.5.1" > requirements.txt
[:~/test2] [test-pot] master ± cat requirements.txt
matplotlib==2.0.2
> numpy==1.13.1
> networkx==1.11
> scikit-learn==0.18.2
> scipy==0.19.1
> pandas==0.19.1
> POT==0.5.1
[:~/test2] [test-pot] master ± cat requirements.txt
matplotlib==2.0.2
numpy==1.13.1
networkx==1.11
scikit-learn==0.18.2
scipy==0.19.1
pandas==0.19.1
POT==0.5.1
[:~/test2] [test-pot] master ± pip install -r requirements.txt
Collecting matplotlib==2.0.2 (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/60/d4/6b6d8a7a6bc69a1602ab372f6fc6e88ef88a8a96398a1a25edbac636295b/matplotlib-2.0.2-cp36-cp36m-manylinux1_x86_64.whl
Collecting numpy==1.13.1 (from -r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/59/e2/57c1a6af4ff0ac095dd68b12bf07771813dbf401faf1b97f5fc0cb963647/numpy-1.13.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting networkx==1.11 (from -r requirements.txt (line 3))
Using cached https://files.pythonhosted.org/packages/d3/2c/e473e54afc9fae58dfa97066ef6709a7e35a1dd1c28c5a3842989322be00/networkx-1.11-py2.py3-none-any.whl
Collecting scikit-learn==0.18.2 (from -r requirements.txt (line 4))
Using cached https://files.pythonhosted.org/packages/d1/51/ca5be39c576c981cf8b8359fcb1ee49ca43d59b833415205cec5ef5c0fcd/scikit_learn-0.18.2-cp36-cp36m-manylinux1_x86_64.whl
Collecting scipy==0.19.1 (from -r requirements.txt (line 5))
Using cached https://files.pythonhosted.org/packages/0e/46/da8d7166102d29695330f7c0b912955498542988542c0d2ae3ea0389c68d/scipy-0.19.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting pandas==0.19.1 (from -r requirements.txt (line 6))
Collecting POT==0.5.1 (from -r requirements.txt (line 7))
Collecting cycler>=0.10 (from matplotlib==2.0.2->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib==2.0.2->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl
Collecting python-dateutil (from matplotlib==2.0.2->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.0,!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 (from matplotlib==2.0.2->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/de/0a/001be530836743d8be6c2d85069f46fecf84ac6c18c7f5fb8125ee11d854/pyparsing-2.3.1-py2.py3-none-any.whl
Collecting six>=1.10 (from matplotlib==2.0.2->-r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting decorator>=3.4.0 (from networkx==1.11->-r requirements.txt (line 3))
Using cached https://files.pythonhosted.org/packages/f1/cd/7c8240007e9716b14679bc217a1baefa4432aa30394f7e2ec40a52b1a708/decorator-4.3.2-py2.py3-none-any.whl
Collecting cython (from POT==0.5.1->-r requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/e1/fd/711507fa396064bf716493861d6955af45369d2c470548e34af20b79d4d4/Cython-0.29.6-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: six, cycler, pytz, python-dateutil, pyparsing, numpy, matplotlib, decorator, networkx, scikit-learn, scipy, pandas, cython, POT
Successfully installed POT-0.5.1 cycler-0.10.0 cython-0.29.6 decorator-4.3.2 matplotlib-2.0.2 networkx-1.11 numpy-1.13.1 pandas-0.19.1 pyparsing-2.3.1 python-dateutil-2.8.0 pytz-2018.9 scikit-learn-0.18.2 scipy-0.19.1 six-1.12.0
And it automatically installed cython which seemed to be the bug in what you sent us. I tried it with python3.6 and python2.7 and it worked.
I'm not familiar with virtualenvwrapper but i don't this the discrepancy come from here. So it might be a problem of version from pip? The install_requires
parameter has only been added recently and if you have an old pip he might discard it and not install cython that is required to compile POT. if you want to be sure to compile whatever the pip version you can always add cython in the requirement (I know nobody wants that but we do not provide pre-compiled eggs on pipy).
Edit: reran the test after cleaning PYTHONPATH variable.
from pot.
@rflamary thank you for looking into this. I don't think it's the version of pip since we are replicating the bug across a number of computers.
from pot.
OK I managed to reproduce the problem.
This occurs on python2.7 when i force it for the virtual environment
virtualenv --python=/usr/bin/python2.7 test_pot_py27
It works on python 3.6...
I'm wondering if it is a problem related to pypa/pip#2478
Its the same pip version for both, this is a weird bug. install_requires
should work on both 2.7 and 3.6 but it seems to fail installing Cython before POT on 2.7...
edit: wrong link
from pot.
Here's a Dockerfile that shows the package failing using pip install with the error File "/tmp/pip-build-y26thcux/POT/setup.py", line 7, in
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'
FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install python3.7 python3-pip build-essential
RUN pip3 install POT
from pot.
This is perfect for debug thank you.
I will have a look (probably after the NeurIPS deadline) but i think it might be corrected by properly defining build dependencies as discussed https://packaging.python.org/specifications/declaring-build-dependencies/ and https://www.python.org/dev/peps/pep-0518/.
from pot.
Related Issues (20)
- How to conduct scenario reduction with this project?
- Too low tolerance in `test_solve_sample_methods`
- Upstream Licencing conflict with GPL: POT and CVXOPT HOT 3
- Issue with Importing POT Library in Apple Silicon Environment with TensorFlow HOT 1
- Jax backend: jax.errors.TracerArrayConversionError HOT 3
- Numpy 2.0 compatibility HOT 11
- Questions for ot.barycenter HOT 1
- Deprecate distutils in favor of setuptools
- Add task runners for tests and extend contributing docs
- [Bug] Linesearch hidden in scipy 1.14 HOT 1
- Incompatibility with numpy 2.0 HOT 2
- Mean computed without weights in empirical gaussain OT
- UnbalancedSinkhorn Transport fails to transform due to "nx.array_equal"
- question regarding applying transport
- How is transport plan (couplings) retrieved from `ot.da.LinearGWTransport()`?
- Incorrect computation of `cost_correction` matrix in `ot.da.EMDTransport` HOT 2
- Wasserstein discriminant analysis (WDA) with stochastic gradient descent HOT 1
- Self distances with ot.bregman.empirical_sinkhorn2 higher than expected
- issue with using other metric choices with `emd2_1d` HOT 2
- p-norm vs Minkowski distance
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pot.