Comments (9)
That's the mini batch normalization. I think I got this implementation from OpenAI. If you get only ones, it probably means your features are close to 0 (exp(0) == 1).
from deeplearningimplementations.
diffs = K.expand_dims(x, 3) - K.expand_dims(K.permute_dimensions(x, [1, 2, 0]), 0)
the former is equal to the succedent, so diffs= 0
, no matter what is inputted. Maybe this is a bug.
from deeplearningimplementations.
Can you give me a link to OpenAI implementation? Thanks a lot.
from deeplearningimplementations.
https://github.com/openai/improved-gan/blob/master/mnist_svhn_cifar10/nn.py
from deeplearningimplementations.
I don't think diffs should always be 0 unles the transpose of x is equal to x somehow.
from deeplearningimplementations.
How can K.permute_dimensions(x, [1, 2, 0])
realize the function of transpose of x? Can you expand it in detail? Thank you very much. This confuses me a lot!
from deeplearningimplementations.
It just switches the axes.
In 2D it is the transpose function you're used to, a nxm matrix turns into an mxn matrix.
In >2D it is harder to visualize but in this case, an (a, b, c) matrix becomes a (b, c, a) matrix.
Then, broadcasting rules are applied in diffs (i.e. the matrices are repeated along the axes with dimension = 1 to allow the subtraction operation)
from deeplearningimplementations.
Genius you are! Is there any formula which can prove the differ between x and tranpose of x calculates mini-batch distance?
from deeplearningimplementations.
You should have a look at the paper (cf. OpenAI Improved techniques to train GAN). They define a way to measure distance within samples. There are of course other ways to do so.
from deeplearningimplementations.
Related Issues (20)
- Can you provide test code?
- WGAN-GP update formula wrong? HOT 2
- tf.sqrt in WGAN-GP gradient penalty formula throws NaNs HOT 2
- Densenet: takes at least 3 arguments (4 given) HOT 2
- Densenet: growth_rate
- Discriminator Implementation Error HOT 1
- CelebA dataset is not accessible HOT 2
- Wasserstein distance formula
- concat_axis missing for the transition layer call inside Densenet HOT 2
- KeyError: '__flags'
- InfoGAN: wrong argument descriptions in help
- densenet on cifar10
- Training tends to yellow colors
- main.py error HOT 1
- Colorful - Yellow results
- How to load my local dataset instead of the Cifar10 dataset and feed it to the DenseNet
- deconv forward_pass
- Quantitative evaluations verified?
- Explaning discriminator model
- How to generate images using model.predict just as the validation ones?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeplearningimplementations.