Keras speed! about capsnet-keras HOT 4 CLOSED

@wensdong Thanks for asking. Firstly, I didn't test the speed of naturomics/CapsNet-Tensorflow. I only test my code on a single GTX 1070 GPU card and the running speed is about 120s / epoch. Then, if it is true about your speed test, I do not think it's due to Keras. Maybe it's because the different implementation of routing algorithm (I used pure matrix computation, naturomics used iteration for capsules in the next layer), though I didn't test it.
At last, I'll give free publicity for Keras. It is a higher level API built up on TF/Theano/CNTK. It is very easy to use. From TensorFlow 1.4, keras has been added to the core code, you can use keras by from tensorflow import keras. I have adapted the Keras code to tensorflow.keras which can be found here:
CapsNet-tf1.4-keras.zip

from capsnet-keras.

@wensdong I didn't use other loops.

• I just tested the two version of inputs_hat computation. It shows the V2 is faster than V1 by 50s.
• The 3 iterations in the routing algorithm is inevitable. num_routing=3 (117s) takes about 12s more than num_routing=1 (105s).
• Other parts are not tested. But I believe that any explicit iterations should be avoided as much as possible.

inputs_hat computation V1

inputs_hat computation V2

PS. A little trick: the progress bar can show normally by enlarging the terminal window.

from capsnet-keras.

Are you sure that both cases were running on the same cpu/gpu ? That's often an explanation for huge difference in training speed.

from capsnet-keras.

Hi Jesff,

I run both codes on same P2 instance. The Naturomics' run about 10+ minutes per epoch, while the Xifeng's run about 2+ minutes per epoch. I did it twice to make sure I was not confused something. I actually tried P3 instance for Naturomics's with no luck, then I started to try others.

Following screenshots for both code executions.

@ Xifeng,

The speed issue normally caused by loops, vectorization normally speed up things very quickly. I did not find any loops in your code except the routing part (funny is that version 2 is indeed use loops, and appear to be faster. Please find below screenshot). Is there any other places you did loops implicitly?

Regards,

from capsnet-keras.