Comments (8)
madsbk
commented on June 20, 2024
1
cc. @charlesbluca
from kvikio.
vyasr
commented on June 20, 2024
1
I would recommend moving unrelated discussions to a new issue. @Weigaa if you need help please open a separate issue.
from kvikio.
Weigaa
commented on June 20, 2024
Something wrong with C++(build from source), After building and cmake, when I make the project, error occurs:
Is this kvikio's fault?
@madsbk @charlesbluca
from kvikio.
Weigaa
commented on June 20, 2024
After adjusting my Gcc and G++ Version both to 11.1.0, and my cmake version is 3.22.5, it can be built.
test basic.io results are:
(wjpytorch) root@nuosen:/mnt/optane/wjtest/kvikio/cpp/examples/examples# ./basic_io
KvikIO defaults:
Compatibility mode: disabled
DriverProperties:
Version: 0.0
Allow compatibility mode: true
Pool mode - enabled: false, threshold: 4 kb
Max pinned memory: 33554432 kb
Write: 4096
Read: 4096
Parallel write (16 threads): 4096
Parallel write (16 threads): 4096
But when I try to use python -m pip install .
Errors occurs:
ERROR: Failed building wheel for kvikio
Failed to build kvikio
ERROR: Could not build wheels for kvikio which use PEP 517 and cannot be installed directly
from kvikio.
Weigaa
commented on June 20, 2024
My pip version is:
pip-22.1.2
more detailed error information are:
(wjpytorch) root@nuosen:/mnt/optane/wjtest/kvikio/python# python -m pip install .
Processing /mnt/optane/wjtest/kvikio/python
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /root/anaconda3/envs/wjpytorch/lib/python3.9/site-packages (from kvikio==22.8.0a0+4.gab99911) (1.22.4)
Building wheels for collected packages: kvikio
Building wheel for kvikio (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for kvikio (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [97 lines of output]
/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/setuptools/dist.py:531: UserWarning: Normalizing '22.08.00a+4.gab99911' to '22.8.0a0+4.gab99911'
warnings.warn(tmpl.format(**locals()))
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 11.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/optane/wjtest/kvikio/python/_cmake_test_compile/build
CMake Error at /tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/skbuild/resources/cmake/FindCython.cmake:65 (message):
Command "/tmp/pip-build-env-gttnfxs0/overlay/bin/cython;--version" failed
with output:
Call Stack (most recent call first):
CMakeLists.txt:38 (find_package)
[1/9] Creating directories for 'rapids-cmake-populate'
[1/9] Performing download step (git clone) for 'rapids-cmake-populate'
-- Avoiding repeated git clone, stamp file is up to date: '/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-subbuild/rapids-cmake-populate-prefix/src/rapids-cmake-populate-stamp/rapids-cmake-populate-gitclone-lastrun.txt'
[2/9] Performing update step for 'rapids-cmake-populate'
fatal: unable to access 'https://github.com/rapidsai/rapids-cmake.git/': Empty reply from server
CMake Error at /mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-subbuild/rapids-cmake-populate-prefix/tmp/rapids-cmake-populate-gitupdate.cmake:97 (execute_process):
execute_process failed command indexes:
1: "Child return code: 128"
FAILED: rapids-cmake-populate-prefix/src/rapids-cmake-populate-stamp/rapids-cmake-populate-update /mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-subbuild/rapids-cmake-populate-prefix/src/rapids-cmake-populate-stamp/rapids-cmake-populate-update
cd /mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-src && /tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/cmake/data/bin/cmake -P /mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-subbuild/rapids-cmake-populate-prefix/tmp/rapids-cmake-populate-gitupdate.cmake
ninja: build stopped: subcommand failed.
CMake Error at /tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.22/Modules/FetchContent.cmake:1087 (message):
Build step for rapids-cmake failed: 1
Call Stack (most recent call first):
/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.22/Modules/FetchContent.cmake:1216:EVAL:2 (__FetchContent_directPopulate)
/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.22/Modules/FetchContent.cmake:1216 (cmake_language)
/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.22/Modules/FetchContent.cmake:1259 (FetchContent_Populate)
_skbuild/linux-x86_64-3.9/cmake-build/RAPIDS.cmake:35 (FetchContent_MakeAvailable)
/mnt/optane/wjtest/kvikio/cpp/cmake/fetch_rapids.cmake:17 (include)
/mnt/optane/wjtest/kvikio/cpp/CMakeLists.txt:17 (include)
-- Configuring incomplete, errors occurred!
See also "/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeOutput.log".
See also "/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeError.log".
File "/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 637, in setup
env = cmkr.configure(
File "/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 328, in configure
raise SKBuildError(
--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build
Command:
cmake /mnt/optane/wjtest/kvikio/python -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_VERSION_STRING:STRING=3.9.12 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython3_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython3_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPython_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPYTHON_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPYTHON_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPYTHON_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
Traceback (most recent call last):
An error occurred while configuring with CMake.
Command:
cmake /mnt/optane/wjtest/kvikio/python -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_VERSION_STRING:STRING=3.9.12 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython3_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython3_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPython_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPYTHON_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPYTHON_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPYTHON_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-s1qlleh3/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
Source directory:
/mnt/optane/wjtest/kvikio/python
Working directory:
/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build
Please see CMake's output for more information.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for kvikio
Failed to build kvikio
ERROR: Could not build wheels for kvikio, which is required to install pyproject.toml-based projects
from kvikio.
Weigaa
commented on June 20, 2024
After dealing with network problem， the errors are:
Processing /mnt/optane/wjtest/kvikio/python
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy in /root/anaconda3/envs/wjpytorch/lib/python3.9/site-packages (from kvikio==22.8.0a0+4.gab99911) (1.22.4)
Building wheels for collected packages: kvikio
Building wheel for kvikio (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for kvikio (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [78 lines of output]
/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/setuptools/dist.py:531: UserWarning: Normalizing '22.08.00a+4.gab99911' to '22.8.0a0+4.gab99911'
warnings.warn(tmpl.format(**locals()))
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 11.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/optane/wjtest/kvikio/python/_cmake_test_compile/build
CMake Error at /tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/skbuild/resources/cmake/FindCython.cmake:65 (message):
Command "/tmp/pip-build-env-gttnfxs0/overlay/bin/cython;--version" failed
with output:
Call Stack (most recent call first):
CMakeLists.txt:38 (find_package)
CMake Error at _skbuild/linux-x86_64-3.9/cmake-build/_deps/rapids-cmake-src/rapids-cmake/cpm/find.cmake:152 (CPMFindPackage):
Unknown CMake command "CPMFindPackage".
Call Stack (most recent call first):
cmake/thirdparty/get_nvcomp.cmake:19 (rapids_cpm_find)
cmake/thirdparty/get_nvcomp.cmake:40 (find_and_configure_nvcomp)
cmake/CMakeLists.txt:15 (include)
-- Configuring incomplete, errors occurred!
See also "/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeOutput.log".
See also "/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build/CMakeFiles/CMakeError.log".
File "/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 637, in setup
env = cmkr.configure(
File "/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 328, in configure
raise SKBuildError(
--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build
Command:
cmake /mnt/optane/wjtest/kvikio/python -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_VERSION_STRING:STRING=3.9.12 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython3_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython3_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPython_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPYTHON_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPYTHON_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPYTHON_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
Traceback (most recent call last):
An error occurred while configuring with CMake.
Command:
cmake /mnt/optane/wjtest/kvikio/python -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-install -DPYTHON_VERSION_STRING:STRING=3.9.12 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPython3_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython3_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython3_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPython_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPython_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPython_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DPYTHON_EXECUTABLE:FILEPATH=/root/anaconda3/envs/wjpytorch/bin/python -DPYTHON_INCLUDE_DIR:PATH=/root/anaconda3/envs/wjpytorch/include/python3.9 -DPYTHON_LIBRARY:PATH=/root/anaconda3/envs/wjpytorch/lib/libpython3.9.so -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-k5sn1ouq/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
Source directory:
/mnt/optane/wjtest/kvikio/python
Working directory:
/mnt/optane/wjtest/kvikio/python/_skbuild/linux-x86_64-3.9/cmake-build
Please see CMake's output for more information.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for kvikio
Failed to build kvikio
ERROR: Could not build wheels for kvikio, which is required to install pyproject.toml-based projects
from kvikio.
madsbk
commented on June 20, 2024
@Weigaa are you running in a Conda environment?
from kvikio.
madsbk
commented on June 20, 2024
Implemented in #70
from kvikio.
Related Issues (20)
- Track Zarr-Python Integration
- Consider using a 80-120% range for `nvcomp` test value sizes.
- Relax NumPy & CuPy dependency in nvCOMP
- Can't find libcufile.so.1 HOT 3
- [Question] kvikIO out-of-the-box support for S3 reads HOT 2
- Include Zarr example/notebook
- Cannot Install Kvikio using conda HOT 2
- [Question] CuFile implementation with RDMA HOT 5
- Crash/segfault on exit when running libcudf tests with kvikIO and CUDA 11.8
- KvikIO: CUDA 12 Conda Packages HOT 1
- Moving from `pynvml` to `nvidia-ml-py`
- Optimize small reads and writes HOT 2
- [FEATURE] Connect KvikIO to a File Descriptior HOT 1
- Use `rapids-cython` & other more recent best practices HOT 3
- why is compat mode faster than gpudirect read with the given python example? HOT 5
- KvikIO: Need to canonicalize dlopen'd library names HOT 5
- Tracking support of new cuFile features HOT 2
- `numpy._typing` requires numpy>=1.23
- Work around scikit-build `include_package_data` bug in `legate/setup.py`
- Basic HDF5 support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kvikio.