Hi,

I think there's an error in your KLD calculation.

This is what you wrote:

# see Appendix B from VAE paper:
# Kingma and Welling. Auto-Encoding Variational Bayes. ICLR, 2014
# https://arxiv.org/abs/1312.6114
# 0.5 * sum(1 + log(sigma^2) - mu^2 - sigma^2)
KLD = -0.5 * torch.mean(torch.mean(1 + logvar - mu.pow(2) - logvar.exp(), 1))

Instead of (what I think it should be)

KLD = -0.5 * torch.mean(torch.sum(1 + logvar - mu.pow(2) - logvar.exp(), 1))

Let me know if I'm right.

Also, could you explain me why you multiply KLD by 0.1 ?
Is that same as multiply BCE a big number? say 1000 for eg?

Could point out to me why you chose beta1 of adam as .5?

Is there a reason why you use normal relu for encoding, but a leaky relu for decoding?