We cannot access this web link: Gaussian Mixture VAE: Lessons in Variational Inference, Generative Models, and Deep Nets

In /pytorch/model/GMVAE.py line 77, I see that the categorical loss, as defined in the file, is the negative entropy minus some constant. I find this confusing for two reasons:

• This loss term is not mentioned in the original paper of the Gumbel-Softmax distribution (https://arxiv.org/pdf/1611.01144.pdf).

• The constant term at the end of this loss can be omitted, right? It is a constant and it does not depend on network parameters. What is the motivation for this term?

I hope some comments can be added to this line of code. I can also help, too!

Since this code was created in tensorflow v1, I cannot run it on my tensorflow v2 system.

Therefore, I need to build a new environment that can run this code. The readme lists the dependencies below:

python == 3.6.8
tensorflow == 1.13.1
numpy == version not listed
matplotlib == version not listed
scipy == version not listed

Other dependencies which are not listed:
keras
scikit-learn

Anyway, this is the system that I built to run this code. I'm posting this so that others can build a compatible system since installing old versions of packages and making sure they are compatible is a pain.

python == 3.6.8
tensorflow == 1.14
numpy == 1.19.5
matplotlib == 2.2.2
scipy == 1.5.0
keras == 2.3.1
scikit-learn == 0.23.2

That being said, thank you very much for providing this resource. I was finally able to run the code after installing all the versions I listed above, and it ran smoothly.

Here the prob-cat/true_labels are obtained from predicted logits. How is this considered as true labels in loss function

Any help would be much appreciated. Thanks

Hello,

thanks for the nice implementation. I am having a hard time understanding the use of entropy loss in the GMVAE. why does it take the targets as real: (array) corresponding array containing the true labels? If I understood this correctly, there is no use for the true labels in the GMVAE training after all. Your help is much appreciated on this!

Hi @jariasf ,
Thanks for the code, first of all. I was wondering, does the GMVAE model definition in your repo here: https://github.com/jariasf/GMVAE/blob/master/pytorch/networks/Networks.py
essentially implement the "Real Gaussian Mixture VAE" as described in this blog post?: http://ruishu.io/2016/12/25/gmvae/
It seems so, but the readme.md doesn't explicitly say that, so I wanted to confirm.

Thanks in advance!

Hi @jariasf,

Thanks for your nice implementation. I was wondering whether you have implemented this method on Cifar10? Have you ever tried? I did and did not receive any improvement for clustering accuracy. What I did was to add a resnet18 feature extractor as the backbone to extract feature x (takes an image and extract x and then put through the network named py_x) and also a deconvolutional neural network as the decoder which took as the input z and generated the reconstructed image. Do you have any idea?

Best,
Mohammad