AutoML Video Edge Library is an open source engine used for inferencing models trained using AutoML Video. It supports running Tensorflow, TF-TRT, TFLite, and EdgeTPU-optimized TFLite models.
I'm Developing For:
If you are looking to do inferencing with no additional hardware, using only CPU then you may use the vanilla Tensorflow (.pb) and TFLite (.tflite) models.
sudo apt-get update
sudo apt-get install python3.7
sudo apt-get install python3-pip
pip3 install opencv-contrib-python --user
pip3 install numpy
Note: opencv-contrib-python is only necessary for the examples, but can be excluded if only the library is being used.
If you plan on running TFLite models on the desktop, install the TFLite interpreter: https://www.tensorflow.org/lite/guide/python
If you plan on running Tensorflow models on desktop:
pip3 install tensorflow==1.14
git clone https://github.com/google/automl-video-ondevice
After that is done downloading, move into the directory.
cd automl-video-ondevice
For TFLite:
python3 examples/video_file_demo.py --model=data/traffic_model.tflite
For Tensorflow:
python3 examples/video_file_demo.py --model=data/traffic_model.pb
Make sure you've setup your coral device: https://coral.ai/docs/setup
Install the TFLite runtime on your device: https://www.tensorflow.org/lite/guide/python
sudo apt-get update
sudo apt-get install git
sudo apt-get install python3-opencv
pip3 install numpy
git clone https://github.com/google/automl-video-ondevice
After that is done downloading, move into the directory.
cd automl-video-ondevice
python3 examples/video_file_demo.py --model=data/traffic_model_edgetpu.tflite
sudo apt-get update
sudo apt-get install git
sudo apt-get install python3.7-pip
sudo apt-get install python3-opencv
sudo apt-get install protobuf-compiler libprotoc-dev
sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran
pip3 install cython numpy
pip3 install --no-cache-dir --ignore-installed --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v44 'tensorflow<2'
git clone https://github.com/NVIDIA/TensorRT.git
cd oss/tools/onnx-graphsurgeon
make install
pip3 install --ignore-installed git+https://github.com/onnx/tensorflow-onnx
git clone https://github.com/google/automl-video-ondevice
After that is done downloading, move into the directory.
cd automl-video-ondevice
python3 examples/video_file_demo.py --model=data/traffic_model_trt.pb
For TensorRT, the chip Engines converted with must be the same kind of chip to run inference workflow on. We hereby convert a TensorRT Engine on the Nano Platform. Later on, we hope to export the onnx model from Cloud console to hide the onnx converting steps from users.
First convert the model to ONNX format, use the model downloaded from GCP Video Intelligence console:
python3 -m tf2onnx.convert --graphdef frozen_inference_graph.pb --inputs image_tensor:0 --outputs detection_boxes:0,detection_classes:0,detection_scores:0,num_detections:0 --output models/model_opset_10.onnx --fold_const --opset 10 --verbose
then,
python3 examples/modify_onnx_model_opset10.py --onnx model_opset_10.onnx --modified model_opset_10_modified.onnx
Then convert ONNX graph to a TensorRT Engine file, please note the TensorRT conversion is a memory intensive process so be mindful when running on less powerful device like Nano:
/usr/src/tensorrt/bin/trtexec --onnx=model_opset_10_modified.onnx --fp16 --workspace=2048 --allowGPUFallback --saveEngine=vot_fp16.plan --verbose &> trtexec_build.log
The log could be viewed by tail -f trtexec_build.log. The TensorRT Engine file vot_fp16.plan could be deployed to a Deep Stream application.
![copybara-service[bot] avatar](https://avatars.githubusercontent.com/in/44061?v=4 "copybara-service[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent