snf / keras-fractalnet Goto Github PK
View Code? Open in Web Editor NEWFractalNet implementation in Keras: Ultra-Deep Neural Networks without Residuals
License: MIT License
FractalNet implementation in Keras: Ultra-Deep Neural Networks without Residuals
License: MIT License
Hello,
How many times is the fractalnet built in this code? From the paper, it seems that it should be built numerous times, but is that how this code is working?
Also, what exactly are all of the dependencies for this code as I'm having issues getting it to run with the current versions of keras and tesnorflow I have installed. Also, I've been trying to use Python3 but I'm not sure if it is supposed to be Python3.
I have taken a deeper look into your FractalNet implementation, but there is one thing I don't understand. Your global_path_arr
is initialized with a (seeded) numpy random array. This means that during training it will be generated once and then not changed.
The fact that you seed the np.random
at the beginning means that even if the random function was called over and over it would always produce the same results.
I tried to implement FractalNet on Cifar from scratch using your implementation as reference and I have run into the same problem, how to generate one path configuration for drop-path per batch.
Could you clarify if this is a bug or if I missed something entirely?
I am trying to run the CIFAR-10 example using the default setup/values with the only exception that I use verbose=1
when fitting. The printed loss is always NaN
and never seems to budge. There is minor fluctuation in the accuracy but no improvement. I have tried this with Adam and SGD optimizers with same results.
I'm able to reproduce this on GPU and CPU. I'm using bleeding-edge Theano and Keras' current (as of writing).
I run the code with the lastest tensorflow and python3, only modifying a few lines. However, I also enconunter the same problem as mentioned in #3 , I tried solving this problemm following the direction described in #2 , no effect. Afterward, I carefully check all the code and found that when you add the JoinLayer, the parameter 'global_switch' and 'global_path' are set to fixed values, which means that although this model owns five JoinLayers, all these JoinLayers have different fixed global_switch and global_path respectively. And after these JoinLayers joined to tf Graph, they can only use the fixed values, not flexible values. So the global droppath will take no effect, so will the K,swtich in the function _drop_path(self, inputs). I suggest that the parameter 'global_switch' and 'global_path' can be initialized in the JoinLayer so that they can take effect. @snf
So I've been training for a while now (40 epochs) and my accuracy hasn't increased at all (and my loss hasn't decreased either).
I've reproduced this with both Adam and SGD.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.