This is the official implementation of CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices in The 20th Annual International Conference on Mobile Systems, Applications and Services, which is a novel framework for co-execution of deep learning models on mobile devices.

CoDL can fully utilize the heterogeneous processors to accelerate each operator of a model, which makes it different from available inference frameworks. CoDL integrates two novel techniques: 1) hybrid-type-friendly data sharing, which allows each processor to use its efficient data type for inference. To reduce data sharing overhead, we also propose hybrid-dimension partitioning and operator chain methods; 2) non-linearity- and concurrency-aware latency prediction, which can direct proper operator partitioning by building an extremely light-weight but accurate latency predictor for different processors.

We evaluate CoDL on a variety of smartphones and deep learning models. The inference speed of CoDL achieves 680ms on RetinaFace, 140ms on YOLOv2, 137ms on VGG-16, 244ms on PoseNet, and 267ms on Fast Style Transfer in our Snapdragon 855 platform (Xiaomi 9).

Below is the list of all the models and their performance on CoDL.

Any questions are welcome. Our paper can be found here.

1. For building execution files, please read and follow the instruction in codl-mobile/README.md.
2. For evaluating in your smartphones, please read and follow the instruction in codl-eval-tools/README.md.

@inproceedings{jia2022codl,
    author = {Jia, Fucheng and Zhang, Deyu and Cao, Ting and Jiang, Shiqi and Liu, Yunxin and Ren, Ju and Zhang, Yaoxue},
    title = {CoDL: Efficient CPU-GPU Co-execution for Deep Learning Inference on Mobile Devices},
    year = {2022},
    publisher = {ACM},
    url = {https://doi.org/10.1145/3498361.3538932},
    doi = {10.1145/3498361.3538932},
    booktitle = {The 20th Annual International Conference on Mobile Systems, Applications and Services (MobiSys '22)},
}