Installation | Usage | Release Notes | Documentation | How to Contribute | License
oneAPI Collective Communications Library (oneCCL) provides an efficient implementation of communication patterns used in deep learning.
oneCCL is integrated into:
- Horovod* (distributed training framework). Refer to Horovod with oneCCL for details.
- PyTorch* (machine learning framework). Refer to PyTorch bindings for oneCCL for details.
oneCCL is part of oneAPI.
- Ubuntu* 18
- GNU*: C, C++ 4.8.5 or higher.
Refer to System Requirements for more details.
Intel(R) oneAPI DPC++/C++ Compiler with Level Zero v1.0 support.
To install Level Zero, refer to the instructions in Intel(R) Graphics Compute Runtime repository or to the installation guide for oneAPI users.
- AVX512F-based implementation requires GCC 4.9 or higher.
- AVX512_BF16-based implementation requires GCC 10.0 or higher and GNU binutils 2.33 or higher.
General installation scenario:
cd oneccl
mkdir build
cd build
cmake ..
make -j install
If you need a clean build, create a new build directory and invoke cmake within it.
You can also do the following during installation:
- Specify installation directory
- Specify the compiler
- Specify
SYCLcross-platform abstraction level - Specify the build type
- Enable
makeverbose output
Use the command:
$ source <install_dir>/env/setvars.sh
$ mpirun -n 2 <install_dir>/examples/benchmark/benchmarkThe ccl-bundled-mpi flag in vars.sh can take values "yes" or "no" to control if bundled Intel MPI should be used or not. Current default is "yes", which means that oneCCL temporarily overrides the mpi implementation in use.
In order to suppress the behavior and use user-supplied or system-default mpi use the following command instead of sourcing setvars.sh:
$ source <install_dir>/env/vars.sh --ccl-bundled-mpi=noThe mpi implementation will not be overridden. Please note that, in this case, user needs to assure the system finds all required mpi-related binaries.
There are two ways to set worker threads (workers) affinity: automatically and explicitly.
- Set the
CCL_WORKER_COUNTenvironment variable with the desired number of workers per process. - Set the
CCL_WORKER_AFFINITYenvironment variable with the valueauto.
Example:
export CCL_WORKER_COUNT=4
export CCL_WORKER_AFFINITY=auto
With the variables above, oneCCL will create four workers per process and the pinning will depend from process launcher.
If an application has been launched using mpirun that is provided by oneCCL distribution package then workers will be automatically pinned to the last four cores available for the launched process. The exact IDs of CPU cores can be controlled by mpirun parameters.
Otherwise, workers will be automatically pinned to the last four cores available on the node.
- Set the
CCL_WORKER_COUNTenvironment variable with the desired number of workers per process. - Set the
CCL_WORKER_AFFINITYenvironment variable with the IDs of cores to pin local workers.
Example:
export CCL_WORKER_COUNT=4
export CCL_WORKER_AFFINITY=3,4,5,6
With the variables above, oneCCL will create four workers per process and pin them to the cores with the IDs of 3, 4, 5, and 6 respectively.
oneCCLConfig.cmake and oneCCLConfigVersion.cmake are included into oneCCL distribution.
With these files, you can integrate oneCCL into a user project with the find_package command. Successful invocation of find_package(oneCCL <options>) creates imported target oneCCL that can be passed to the target_link_libraries command.
For example:
project(Foo)
add_executable(foo foo.cpp)
# Search for oneCCL
find_package(oneCCL REQUIRED)
# Connect oneCCL to foo
target_link_libraries(foo oneCCL)To generate oneCCLConfig files for oneCCL package, use the provided cmake/scripts/config_generation.cmake file:
cmake [-DOUTPUT_DIR=<output_dir>] -P cmake/script/config_generation.cmake
- Optimizing DLRM by using PyTorch with oneCCL Backend
- Intel MLSL Makes Distributed Training with MXNet Faster
- oneAPI, oneCCL and OFI: Path to Heterogeneous Architecure Programming with Scalable Collective Communications: recording and slides
See CONTRIBUTING for more information.
Distributed under the Apache License 2.0 license. See LICENSE for more information.
See SECURITY for more information.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent