can't compile tensorflow .pb model with Sagemaker ENO about neo-ai-dlr HOT 9 OPEN

@Least1924 thanks for reaching out. Would you mind sharing your .pb file? it is weird since inception should be supported well.

Currently, if the model compilation failed, then we will keep the model as it is, dlr then will use the executor of vanilla TF framework. So the the single .pb file in your compiled .tar.gz is expected for this case.

from neo-ai-dlr.

@yongwww Thanks for your prompt reply ! Here is the single .pb file in my compiled.

https://drive.google.com/open?id=1L6Ex98CEvnDe2hM_ZFxotzEZB7IxUzGy

from neo-ai-dlr.

@Least1924 thanks for reaching out. Would you mind sharing your .pb file? it is weird since inception should be supported well.

Currently, if the model compilation failed, then we will keep the model as it is, dlr then will use the executor of vanilla TF framework. So the the single .pb file in your compiled .tar.gz is expected for this case.

@yongwww sorry to bother you, is there any relevant progress on this issue ?

from neo-ai-dlr.

@Least1924 sorry for the late response. Failed to compile with you .pb file in tvm's tf frontend converter. It should be an issue in tf converter, in this case we return the same pb file as your input, the pb file can be supported with dlr. But for sure, we need to fix tf converter to get optimized model.

from neo-ai-dlr.

@yongwww Thank you very much for your work. Please let me know if the issue is fixed.

from neo-ai-dlr.

@Least1924 the main blocker for this issue is your preprocessing stuff in this model, DecodeJpeg can not be handled so far. Could you please move the preprocessing (node DecodeJpeg and DecodeJpeg's input node) out of the model? then just feed the preprocessed data to do inference.

from neo-ai-dlr.

@Least1924 i'm facing same issue with tensorflow ssd model , please let me know how you fixed this

from neo-ai-dlr.

you can use transform_graph tool from tensorflow. It allows to cut sub-graph from your full graph.
Command example is below. You need to set new input and output

transform_graph \
--in_graph=mobilenet_v2_1.0_224_frozen.pb \
--out_graph=mobilenet_v2_1.0_224_opt.pb \
--inputs='input' \
--outputs='MobilenetV2/Predictions/Reshape_1' \
--transforms='
strip_unused_nodes(type=float, shape="1,224,224,3")
remove_nodes(op=Identity, op=CheckNumerics)
fold_constants(ignore_errors=true)
fold_batch_norms
fold_old_batch_norms'

from neo-ai-dlr.

I compiled a yolo model from mxnet using sagemaker neo and tested it on the edge device. The performance is really bad. It takes 25 secs for detection in a frame. Similarly, the model from tensorflow has 0.1fps. Is there anything wrong with my compilation? can you please help me with this problem?
Thank you for all your help.

from neo-ai-dlr.