A flexible python front-end inference SDK.
-
Flexible
FlexInfer has a Python front-end, which makes it easy to build a computer vision product prototype.
-
Efficient
Most of time consuming part of FlexInfer is powered by C++ or CUDA, so FlexInfer is also efficient. If you are really hungry for efficiency and don't mind the trouble of C++, you can refer to CheetahInfer.
This project is released under Apache 2.0 license.
- Linux
- Python 3.6 or higher
- TensorRT 7.1.3.4 or higher
- PyTorch 1.4.0 or higher
- CUDA 10.2 or higher
- volksdep 3.2.0 or higher
We have tested the following versions of OS and softwares:
- OS: Ubuntu 16.04.6 LTS
- Python 3.6.9
- TensorRT 7.1.3.4
- PyTorch 1.6.0
- CUDA: 10.2
- volksdep: 3.2.0
- If your platform is x86 or x64, you can create a conda virtual environment and activate it.
conda create -n flexinfer python=3.6.9 -y
conda activate flexinfer
-
Install volksdep following the official instructions
-
Setup
pip install "git+https://github.com/Media-Smart/flexinfer.git"
We provide some examples for different tasks.
- Device: Jetson AGX Xavier
- CUDA: 10.2
Tasks | framework | version | input shape | data type | throughput(FPS) | latency(ms) |
Classification (Resnet18) | pytorch | 1.5.0 | (1, 3, 224, 224) | fp16 | 172 | 6.01 |
tensorrt | 7.1.0.16 | (1, 3, 224, 224) | fp16 | 754 | 1.8 | |
Segmentation(Unet) | pytorch | 1.5.0 | (1, 3, 513, 513) | fp16 | 15 | 63.27 |
tensorrt | 7.1.0.16 | (1, 3, 513, 513) | fp16 | 29 | 34.03 | |
Object Detection(RetinaNet-Resnet50) | pytorch | 1.5.0 | (1, 3, 768, 1280) | fp16 | 8 | 118.79 |
tensorrt | 7.1.0.16 | (1, 3, 768, 1280) | fp16 | 15 | 68.10 | |
Scene Text Recognition (ResNet-CTC) | pytorch | 1.5.0 | (1, 1, 32, 100) | fp16 | 113 | 10.75 |
tensorrt | 7.1.0.16 | (1, 1, 32, 100) | fp16 | 308 | 3.55 |
We provide some toolboxes of different tasks for training, testing and deploying.
-
Classification (vedacls)
-
Segmentation (vedaseg)
-
Object Detection (vedadet)
-
Scene Text Recognition (vedastr)
This repository is currently maintained by Yuxin Zou (@Yuxin Zou), Jun Sun(@ChaseMonsterAway), Hongxiang Cai (@hxcai) and Yichao Xiong (@mileistone).