TensorFlow inference is quite slow inside DetectionSuite compared to that on ObjetcDetector about detectionmetrics HOT 6 CLOSED

Hi @naxvm and @jmplaza ,
Could you please share the network, of which statistics have been mentioned, so that I can test the same and report it on my end.

from detectionmetrics.

Hello @vinay0410

The network file which I've used was the v2. SSD Mobilenet here is the link. I've used it on GPU (not so fast one, but it makes 125 ms on objectdetector), inferring frames from a video in DetectionSuite. Could you give it a check? Thanks!!

from detectionmetrics.

@vinay0410 I'm doing a fast test of the inference app. As I can see on the TensorFlow output, a new tf.Session() is being called to process each frame. I suspect that this could be guilty for that huge timing, as the network's first prediction is much slower (in objectdetector I make a first dummy inference before starting the on-flight process). Could you check this, or guide about where is the Python-TF code you are using? Thanks!!

from detectionmetrics.

As I mentioned, a single Session instance is created for each frame on the inference file.

This session should be persistent across the evaluated frames (that's the main reason why I created the DetectionNetwork class on objectdetector.

If the C++ interface allows you to create objects, feel free to use the class. For what I've seen, it's ready to go here (simple functions renaming). I am pretty sure this is the main reason which is bottlenecking us...

from detectionmetrics.

Hi @naxvm and @jmplaza ,
I tested the same network on DetectionSuite which is SSD MobileNet v2 COCO, and the inference speed I am getting is 150ms on CPU.
I think that you are still using a previous version of DetectionSuite, please try updating it.
I have also made a video proving the same fact, using updated DetectionSuite.
Link to Video

Also, I have already inferred a dummy tensor here.
And the tf.Session() variable was called only once and then stored here in the object for further use by its methods.

I suggest updating the code with a fresh pull.

Thanks

from detectionmetrics.

Hello Vinay,

Sorry, it was my fault. I was on an old commit (I thought I was updated). Both TensorFlow and Keras inferences are working properly, similar timings than on objectdetector.

Closing this, thanks!

from detectionmetrics.