snf / keras-fractalnet Goto Github PK

I have taken a deeper look into your FractalNet implementation, but there is one thing I don't understand. Your global_path_arr is initialized with a (seeded) numpy random array. This means that during training it will be generated once and then not changed.

The fact that you seed the np.random at the beginning means that even if the random function was called over and over it would always produce the same results.

I tried to implement FractalNet on Cifar from scratch using your implementation as reference and I have run into the same problem, how to generate one path configuration for drop-path per batch.

Could you clarify if this is a bug or if I missed something entirely?

Hello,

How many times is the fractalnet built in this code? From the paper, it seems that it should be built numerous times, but is that how this code is working?

Also, what exactly are all of the dependencies for this code as I'm having issues getting it to run with the current versions of keras and tesnorflow I have installed. Also, I've been trying to use Python3 but I'm not sure if it is supposed to be Python3.

I run the code with the lastest tensorflow and python3, only modifying a few lines. However, I also enconunter the same problem as mentioned in #3 , I tried solving this problemm following the direction described in #2 , no effect. Afterward, I carefully check all the code and found that when you add the JoinLayer, the parameter 'global_switch' and 'global_path' are set to fixed values, which means that although this model owns five JoinLayers, all these JoinLayers have different fixed global_switch and global_path respectively. And after these JoinLayers joined to tf Graph, they can only use the fixed values, not flexible values. So the global droppath will take no effect, so will the K,swtich in the function _drop_path(self, inputs). I suggest that the parameter 'global_switch' and 'global_path' can be initialized in the JoinLayer so that they can take effect. @snf

I am trying to run the CIFAR-10 example using the default setup/values with the only exception that I use verbose=1 when fitting. The printed loss is always NaN and never seems to budge. There is minor fluctuation in the accuracy but no improvement. I have tried this with Adam and SGD optimizers with same results.

I'm able to reproduce this on GPU and CPU. I'm using bleeding-edge Theano and Keras' current (as of writing).

So I've been training for a while now (40 epochs) and my accuracy hasn't increased at all (and my loss hasn't decreased either).

I've reproduced this with both Adam and SGD.