dkurt / openvino_pytorch_layers Goto Github PK

View Code? Open in Web Editor NEW

26.0 7.0 13.0 88 KB

How to export PyTorch models with unsupported layers to ONNX and then to Intel OpenVINO

Home Page: https://github.com/openvinotoolkit/openvino

License: Apache License 2.0

Python 18.24% CMake 1.35% C++ 80.41%

openvino pytorch unpooling onnx model-optimizer custom intel fft intel-openvino

openvino_pytorch_layers's Introduction

⚠️ Source code will be continued to be supported and developed in OpenVINO contrib. Thanks for all who used.

Repository with guides to enable some layers from PyTorch in Intel OpenVINO:

OpenVINO Model Optimizer extension

To create OpenVINO IR, use extra --extension flag to specify a path to Model Optimizer extensions that perform graph transformations and register custom layers.

mo --input_model model.onnx --extension openvino_pytorch_layers/mo_extensions

Custom CPU extensions

You also need to build CPU extensions library which actually has C++ layers implementations:

source /opt/intel/openvino_2022/setupvars.sh

cd user_ie_extensions
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release && make -j$(nproc --all)

Add compiled extensions library to your project:

from openvino.runtime import Core

core = Core()
core.add_extension('user_ie_extensions/build/libuser_cpu_extension.so')

model = ie.read_model('model.xml')
compiled_model = ie.compile_model(model, 'CPU')

openvino_pytorch_layers's People

Contributors

Stargazers

Watchers

Forkers

alexeyhorkin avbelova samithuang ngaloppo zhaohb ayankumarbhunia dzy130120 likholat jooho-enerzai teddy12155555 wang7393 shengshanbai zhouleidcc

openvino_pytorch_layers's Issues

Support dim parameter for FFT layer

#31

DeformableConvolution

Hi, @likholat! We would really enjoy to see https://gist.github.com/likholat/00c47703cd65a90b40f44fdeb8edbaf4 integrated in this repo.

Question about Standalone Operation

Once I convert the PyTorch model to ONNX -- Can I run the converted model (involving grid_sample) standalone without any dependency on this openvino_pytorch_layers package?

Does the grid_sample operation support other arguments?

Hi, @dkurt
Thanks for your sharing! I wonder whether this op works for other arguments?
For example, F.grid_sample(img, grid, mode='bilinear', padding_mode='reflection', align_corners=True).

Will it be supported in iGPU in the future?

Best regards,
DwenGu

Does grid_sample work for openVINO?

Thanks for your sharing! As it grid_sample is not mentioned in README, I wonder whether it works for openVINO custom layer?

Check FFTShift on odd dimensions

[ ERROR ] There is no registered "infer" function for node "Unpooling_53" with op = "Unpooling". Please implement this function in the extensions.

Hi @dkurt ,
Your solution to overcome a pytorch to IR model conversion issue here was really helpful, however we couldn't proceed with the PR due to dataset license issues. I am facing issue with model conversion in another model, this time it is due to maxunpooling layer in the decoder. I came across your really good extension. However, I am still not able to resolve the problem.

Receiving the following the error while converting model with maxunpool.

[ ERROR ]  Cannot infer shapes or values for node "Unpooling_53".
[ ERROR ]  There is no registered "infer" function for node "Unpooling_53" with op = "Unpooling". Please implement this function in the extensions. 
 For more information please refer to Model Optimizer FAQ, question #37. (https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=37#question-37)
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <UNKNOWN>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'openvino.tools.mo.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "Unpooling_53" node. 
 For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)

Input shape provided in mo is [1,1,512,512].

Environment Specs:

torch==1.12.0
torchvision==0.13.0
numpy==1.19.5
openvino-dev[onnx]==2022.1.0
onnxruntime==1.10.0

You can find the model definition, model weights, onnx file, error and debug log from mo here.

dkurt / openvino_pytorch_layers Goto Github PK

openvino_pytorch_layers's Introduction

OpenVINO Model Optimizer extension

Custom CPU extensions

openvino_pytorch_layers's People

Contributors

Stargazers

Watchers

Forkers

openvino_pytorch_layers's Issues

Support dim parameter for FFT layer

DeformableConvolution

Question about Standalone Operation

Does the grid_sample operation support other arguments?

Does grid_sample work for openVINO?

Check FFTShift on odd dimensions

[ ERROR ] There is no registered "infer" function for node "Unpooling_53" with op = "Unpooling". Please implement this function in the extensions.

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent