CV-CUDA is an open-source project that enables building efficient cloud-scale Artificial Intelligence (AI) imaging and computer vision (CV) applications. It uses graphics processing unit (GPU) acceleration to help developers build highly efficient pre- and post-processing pipelines. CV-CUDA originated as a collaborative effort between NVIDIA and ByteDance.

Refer to our Developer Guide for more information on the operators available as of release v0.6.0-beta.

To get a local copy up and running follow these steps.

* partial build, no test module (see Known Limitations)
** full build, including test module
*** samples require driver r535 or later to run and are only officially supported with CUDA 12.

• For GCC versions lower than 11.0, C++17 support needs to be enabled when compiling CV-CUDA.
• The C++ test module cannot build with gcc<11 (requires specific C++-20 features). With gcc-9 or gcc-10, please build with option -DBUILD_TESTS=0
• CV-CUDA Samples require driver r535 or later to run and are only officially supported with CUDA 12.
• Only one CUDA version (CUDA 11.x or CUDA 12.x) of CV-CUDA packages (Debian packages, tarballs, Python Wheels) can be installed at a time. Please uninstall all packages from a given CUDA version before installing packages from a different version.
• Test tarballs (cvcuda-tests-*.tar.xz) need to be unpacked at the root level to find existing tests.

For convenience, we provide pre-built packages for various combinations of CUDA versions, Python versions and architectures here. The following steps describe how to install CV-CUDA from such pre-built packages.

We support two main alternative pathways:

• DEB or Tar archive installation (C++/CUDA Libraries, Headers, Python bindings)
• Standalone Python Wheels (containing C++/CUDA Libraries and Python bindings)

Choose the installation method that meets your environment needs.

• Installation of C++/CUDA libraries (cvcuda-lib*) and development headers (cvcuda-dev*):

tar -xvf cvcuda-lib-0.6.0_beta-<cu_ver>-<arch>-linux.tar.xz
tar -xvf cvcuda-dev-0.6.0_beta-<cu_ver>-<arch>-linux.tar.xz

• Installation of Python bindings (cvcuda-python*)

tar -xvf cvcuda-python<py_ver>-0.6.0_beta-<cu_ver>-<arch>-linux.tar.xz

with <cu_ver> the desired CUDA version, <py_ver> the desired Python version and <arch> the desired architecture

• Installation of C++/CUDA libraries (cvcuda-lib*) and development headers (cvcuda-dev*):

sudo apt-get install -y ./cvcuda-lib-0.6.0_beta-<cu_ver>-<arch>-linux.deb ./cvcuda-dev-0.6.0_beta-<cu_ver>-<arch>-linux.deb

• Installation of Python bindings (cvcuda-python*)

sudo apt-get install -y cvcuda-python<py_ver>-0.6.0_beta-<cu_ver>-<arch>-linux.deb

with <cu_ver> the desired CUDA version, <py_ver> the desired Python version and <arch> the desired architecture

Download the appropriate .whl file for your computer architecture, Python and CUDA version from the release assets of current CV-CUDA release. Release information of all CV-CUDA releases can be accessed here. Once downloaded, execute the pip install command to install the Python wheel. For example:

pip install cvcuda_<cu_ver>-0.6.0b0-cp<py_ver>-cp<py_ver>-linux_<arch>.whl

with <cu_ver> the desired CUDA version, <py_ver> the desired Python version and <arch> the desired architecture

Please note that the Python wheels provided are standalone, they include both the C++/CUDA libraries and the Python bindings.

Follow these instruction to build CV-CUDA from source:

1. Set up your local CV-CUDA repository

   a. Install prerequisites needed to setup up the repository.

   On Ubuntu >= 20.04, install the following packages:
   - git-lfs: to retrieve binary files from remote repository
   
   ```shell
   sudo apt-get install -y git git-lfs
   ```
   

   b. After cloning the repository (assuming it was cloned in ~/cvcuda), it needs to be properly configured by running the init_repo.sh script only once.

   ```shell
   cd ~/cvcuda
   ./init_repo.sh
   ```
   

2. Build CV-CUDA

   a. Install the dependencies required for building CV-CUDA

   On Ubuntu >= 20.04, install the following packages:
   - g++-11: compiler to be used
   - cmake (>= 3.20), ninja-build (optional): manage build rules
   - python3-dev: for python bindings
   - libssl-dev: needed by the testsuite (MD5 hashing utilities)
   
   ```shell
   sudo apt-get install -y g++-11 cmake ninja-build python3-dev libssl-dev
   ```
   
   For CUDA Toolkit, any version of the 11.x or 12.x series should work.
   CV-CUDA was tested with 11.7 and 12.2, thus those should be preferred.
   
   ```shell
   sudo apt-get install -y cuda-11-7
   # or
   sudo apt-get install -y cuda-12-2
   ```
   

   b. Build the project

   ```shell
   ci/build.sh [release|debug] [output build tree path] [-DBUILD_TESTS=1|0] [-DPYTHON_VERSIONS='3.8;3.9;3.10;3.11'] [-DPUBLIC_API_COMPILERS='gcc-9;gcc-11;clang-11;clang-14']
   ```
   
   The default build type is 'release'.
   
   If output build tree path isn't specified, it will be `build-rel` for release
   builds, and `build-deb` for debug.
   
   The library is in `build-rel/lib` and executables (tests, etc...) are in `build-rel/bin`.
   
   The `-DBUILD_TESTS` option can be used to disable/enable building the tests (enabled by default, see Known Limitations).
   
   The `-DPYTHON_VERSIONS` option can be used to select Python versions to build bindings and Wheels for.
   By default, only the default system Python3 version will be selected.
   
   The `-DPUBLIC_API_COMPILERS` option can be used to select the compilers used to check public API compatibility.
   By default, gcc-11, gcc-9, clang-11, and clang-14 is tried to be selected and checked.
   

3. Build Documentation

   a. Install the dependencies required for building the documentation

   On Ubuntu >= 20.04, install the following packages:
   - doxygen: parse header files for reference documentation
   - python3, python3-pip: to install some python packages needed
   - sphinx, breathe, exhale, recommonmark, graphiviz: to render the documentation
   - sphinx-rtd-theme: documenation theme used
   
   ```shell
   sudo apt-get install -y doxygen graphviz python3 python3-pip
   sudo python3 -m pip install sphinx==4.5.0 breathe exhale recommonmark graphviz sphinx-rtd-theme
   ```
   

   b. Build the documentation shell ci/build_docs.sh [build folder]

   Example:
   `ci/build_docs.sh build_docs`
   

4. Build and run Samples

   For instructions on how to build samples from source and run them, see the Samples documentation.

5. Run Tests

   a. Install the dependencies required for running the tests

   On Ubuntu >= 20.04, install the following packages:
   - python3, python3-pip: to run python bindings tests
   - torch: dependencies needed by python bindings tests
   
   ```shell
   sudo apt-get install -y python3 python3-pip
   sudo python3 -m pip install pytest torch
   ```
   

   b. Run the tests

   The tests are in `<buildtree>/bin`. You can run the script below to run all
   tests at once. Here's an example when build tree is created in `build-rel`
   
   ```shell
   build-rel/bin/run_tests.sh
   ```
   

6. Package installers and Python Wheels

   a. Package installers

   Installers can be generated using the following cpack command once you have successfully built the project

   cd build-rel
   cpack .

   This will generate in the build directory both Debian installers and tarballs (*.tar.xz), needed for integration in other distros.

   For a fine-grained choice of what installers to generate, the full syntax is:

   cpack . -G [DEB|TXZ]

   • DEB for Debian packages
   • TXZ for *.tar.xz tarballs.

   b. Python Wheels

   By default during the release build, Python bindings and wheels are created for the available CUDA version and the specified Python version(s). The wheels are stored in build-rel/pythonX.Y/wheel folder, where build-rel is the build directory used to build the release build and X and Y are Python major and minor versions. The built wheels can be installed using pip. For example, to install the Python wheel built for CUDA 12.x, Python 3.10 on Linux x86_64 systems:

   pip install cvcuda_cu12-0.6.0b0-cp310-cp310-linux_x86_64.whl

CV-CUDA is an open source project. As part of the Open Source Community, we are committed to the cycle of learning, improving, and updating that makes this community thrive. However, as of release v0.6.0-beta, CV-CUDA is not yet ready for external contributions.

To understand the process for contributing the CV-CUDA, see our Contributing page. To understand our commitment to the Open Source Community, and providing an environment that both supports and respects the efforts of all contributors, please read our Code of Conduct.

The mkop.sh script is a powerful tool for creating a scaffold for new operators in the CV-CUDA library. It automates several tasks, ensuring consistency and saving time.

1. Operator Stub Creation: Generates no-op (no-operation) operator templates, which serve as a starting point for implementing new functionalities.

2. File Customization: Modifies template files to include the new operator's name, ensuring consistent naming conventions across the codebase.

3. CMake Integration: Adds the new operator files to the appropriate CMakeLists, facilitating seamless compilation and integration into the build system.

4. Python Bindings: Creates Python wrapper stubs for the new operator, allowing it to be used within Python environments.

5. Test Setup: Generates test files for both C++ and Python, enabling immediate development of unit tests for the new operator.

Run the script with the desired operator name. The script assumes it's located in /cvcuda/tools/mkop.

./mkop.sh [Operator Name]

If the script is run from a different location, provide the path to the CV-CUDA root directory.

./mkop.sh [Operator Name] [CV-CUDA root]

NOTE: The first letter of the new operator name is captitalized where needed to match the rest of the file structures.

• Initial Setup: The script begins by validating the input and setting up necessary variables. It then capitalizes the first letter of the operator name to adhere to naming conventions.

• Template Modification: It processes various template files (Public.h, PrivateImpl.cpp, etc.), replacing placeholders with the new operator name. This includes adjusting file headers, namespaces, and function signatures.

• CMake and Python Integration: The script updates CMakeLists.txt files and Python module files to include the new operator, ensuring it's recognized by the build system and Python interface.

• Testing Framework: Finally, it sets up test files for both C++ and Python, allowing developers to immediately start writing tests for the new operator.

CV-CUDA operates under the Apache-2.0 license.

CV-CUDA, as a NVIDIA program, is committed to secure development practices. Please read our Security page to learn more.

CV-CUDA is developed jointly by NVIDIA and ByteDance.