problem of function usage about deeplearningimplementations HOT 9 CLOSED

That's the mini batch normalization. I think I got this implementation from OpenAI. If you get only ones, it probably means your features are close to 0 (exp(0) == 1).

from deeplearningimplementations.

diffs = K.expand_dims(x, 3) - K.expand_dims(K.permute_dimensions(x, [1, 2, 0]), 0)
the former is equal to the succedent, so diffs= 0 , no matter what is inputted. Maybe this is a bug.

from deeplearningimplementations.

Can you give me a link to OpenAI implementation? Thanks a lot.

from deeplearningimplementations.

https://github.com/openai/improved-gan/blob/master/mnist_svhn_cifar10/nn.py

from deeplearningimplementations.

I don't think diffs should always be 0 unles the transpose of x is equal to x somehow.

from deeplearningimplementations.

How can K.permute_dimensions(x, [1, 2, 0]) realize the function of transpose of x? Can you expand it in detail? Thank you very much. This confuses me a lot!

from deeplearningimplementations.

It just switches the axes.
In 2D it is the transpose function you're used to, a nxm matrix turns into an mxn matrix.
In >2D it is harder to visualize but in this case, an (a, b, c) matrix becomes a (b, c, a) matrix.

Then, broadcasting rules are applied in diffs (i.e. the matrices are repeated along the axes with dimension = 1 to allow the subtraction operation)

from deeplearningimplementations.

Genius you are! Is there any formula which can prove the differ between x and tranpose of x calculates mini-batch distance?

from deeplearningimplementations.

You should have a look at the paper (cf. OpenAI Improved techniques to train GAN). They define a way to measure distance within samples. There are of course other ways to do so.

from deeplearningimplementations.