Support for Keras (python) inference should be added about detectionmetrics HOT 11 CLOSED

This issue can be solved simultaneously with #24, the network mechanism is right the same than TF (DetectionNetwork class as well), so we just have to handle the image input (should be as a np array) and output retrieval from any of both component. It should work in the analogous.

from detectionmetrics.

Hi @naxvm,
Exactly, I agree with you!
This problem is very similar to TensorFlow support for inference which has already been added here.
It uses C++ to call Python functions and returns bounding boxes, detection scores,classes, etc, which is further used by Detection suite to label inferences on image.

I am currently thinking of loading keras model weights from h5 file and config from json file, just as mentioned here.

Do have a look.
I will add this feature as soon as I add documentation for TensorFlow support

from detectionmetrics.

Hi @naxvm
Hope you are doing well.
Actually I need to clarify something.

In Tensorflow, frozen inference graphs store everything including model architecture, weights and optimizer state, and therefore it is completely unnecessary to use any config files for the same.

Whereas in keras hdf5 file was initially being used to store only weights. That's why in object detector you are using these python scrips, to generate model architecture.

Is that right ?

But now there is support for storing model architecture, model weights and optimizer state in a single hdf5 file, just like inference graphs. Official Documentation for the same is available here.

Though there is problem finding pre-trained HDF5 models containing everything.
But I can use your python scripts to load the models and then dump everything in a HDF file, so that they can be used further.

But don't you think that would be a better way to go for DetectionSuite ?

Because DetectionSuite must be able to load any model without the need to generate its model architecture separately.

Cheers!

from detectionmetrics.

Hello Vinay!

That's exactly the idea. TF way is far better, as it stores all necessary information to load the complete model.

As you said, both implementations are available in Keras (store only weights or complete model). Of course, the idea is to be able to load the complete model, but the trained models I found on the original source only contain weights. So this is made with the purpose that anyone who wants to use a particular HDF5 file containing weights trained on a dataset, is able to just load it (in fact, I created a function which defines a model and embeds the weights into it).

Anyway, I agree with you about the most comfortable way to load it, so feel free to use the Net/utils/create__model_from_weigths package. It contains a function which returns the compiled model with the weights loaded into it. You can save the model to a complete HDF5 file with model.save(), as I suppose you already know.

That should be the most comfortable way to load a Keras network on DetectionSuite without needing to generate the architecture, as you mention. I can add the model.save() sentence to the package if you want, to make it handier.

Cheers for you!

from detectionmetrics.

By the way, this should be more considered than the JSON format to save a model?

Regards!!

from detectionmetrics.

Hi @naxvm ,
I will try to save some models by using your package Net/utils/create_model_from_weights.py.
And since complete models are not available anywhere on the Internet, we can host some sample complete HD5 models on JdeRobot, and attach their links at Github.

I will also try to train faster_rcnn models and dump them in completer HDF5 file, so that they can be hosted at JdeRobot.

What do you think ?

Cheers!

from detectionmetrics.

Seems like a wonderful idea, and it should be easy to do. Download -> invoke model -> load weights -> save HDF5 model -> to Internet, right? 😉

from detectionmetrics.

Hi,

yes, that is the idea: DetectionSuite dealing with complete network files (weights, model...). Maybe adding to this tool the ability to export complete network files from uncomplete ones (like files only with weigths).

Maybe we can put the enriched pretrained models somewhere at JdeRobot or even on a "pretrained_models" directory inside DetectionSuite's github (depending on their sizes...).

yes @vinay0410, faster_rcnn is a good model to test. The mid-term goal is to support several well known detection networks (ssd, yolo...), test them compare them with DetectionSuite against the same fair test datasets. Let's focus on adding general support for Keras detection networks first.

from detectionmetrics.

Hi,
I have successfully implemented Keras Inferencer in DetectionSuite. Though there is some testing and slight improvements necessary to submit a pull request ( will submit it by tomorrow ). I am working on the same.

I will also make it more generalized. It only supports SSD models ( just like dl-objectdetector for Keras ) for now ( some models are mentioned here ). Since Keras functions in a very different way from TensorFlow, I will take a bit more time to make Keras generalized.

Also, @jmplaza , the file size of complete HDF5 models is above 100MB and github rejects files above 100MB. So, it would be better if these files could be hosted at JdeRobot. I will update links of the same in the Documentation with my pull request.

And, @naxvm , the speed of Keras Inferencer for 300x300 model is around 1.39 seconds for an inference. I tested dl-objectdetector, it also had approximately same speed for Keras. This is slow as compared to TensorFlow. I will be researching more on this topic to improve speed. If you have something useful, please do share.

Sample Video using Keras:

Link to Video

from detectionmetrics.

Hello Vinay,

About the HDF5 issue, I experienced the same with the Keras' model, but the easier way we found was to compress it inside a .zip, and we got a deflation under 100 MB, which allowed us to upload the model to GitHub.

And on Keras's performance, I also had slower results than TF (8 fps vs. 4 fps), although more than a sec seems too high for me. I'm on the GPU version of TF, could this be the problem with the timing?

I'm a little busy these days so I can't spend much time on this component now, but I will go further on the next days.

Thanks Vinay, regards!!!

from detectionmetrics.

Yes, my results are on a CPU, that’s why the difference

from detectionmetrics.