Hybrid Parallelism Global Orchestration

This project is in Alpha Stage: The functionality provided should be considered research grade and the name "HPGO" is expected to change upon stable release

Repository temporarily moved to Lab: https://lab.rongyi.io/research/ml/HPGO until further notice. Previous archive can be downloaded here and here.

The binary package of this library is provided for every release.

(deprecated)

cargo install HPGO

(release only)

pip3 install HPGO

# Import HPGO Python API from HPGO.so
import HPGO
# Construct the Conductor object
c = HPGO.conductor_from_torch_graph_and_seps("./profiles/xlnet/graph.txt", 64, 512, [8, 16])
res = c.py_orchestrate()
print(res)

This project is open sourced under the terms of BSD-3-Clause, details of which can be found in the LICENSE file

If you apply this library to any project and research, please cite this code:

@misc{HPGO,
  author       = {Yi Rong},
  title        = {HPGO, [{Hybrid Parallelism} Global Orchestration]},
  howpublished = {\url{https://github.com/LER0ever/HPGO}},
  year         = {2020}
}

The project contains source code from PipeDream, a Microsoft Research project licensed under the MIT License. Code taken from Pipedream includes a Rust file src/input/torch_graph_py.rs, several profiling data files under the profiles directory, and the contrib directory.

This project also contains profiling data provided by the AlibabaPAI/DAPPLE project. The development of the DAPPLE planner continues here.