bjlkeng / sandbox Goto Github PK

Very informative notebooks. Thanks!
In loss function logx_loss, isn't x_decoded_mean supposed to be _x_decoded_mean?

Hi, bjlkeng,
I read your blog and GitHub code. That's quite helpful for me as a new learner to VAE methods.

 When I was reading the code of semi-VAE m2 model, I found that the unlabeled_vae_loss function was different from Kingma's paper?
 I am not quite sure, but should it be an iteration over all possible y and a sum loss as in the first term of Eq(7) in paper "Semi-supervised Learning with Deep Generative Models"?
 
 I will appreciate it if you can reply to my confusion.

MaskingDense function in the notebook example takes ages. Is it expected?

keras 2.1.1
tensorflow 1.4.1

Hello,

Sorry for sharing the file name but I thought this would be the most convenient way to track. Anyway, under the fit_model function in the notebook, there is a comment that says **_# Repeat the labeled data to match length of unlabeled data_**. I wonder wouldn't approach cause overfitting in the training session. I also saw this type of approach somewhere else but couldn't comprehend how the mechanism works here. I am aware that this is done due to handling the different sized labeled and unlabeled loaders though. I hope this clearly states my concern over here.

Thanks in advance!

First of all, thanks a lot for sharing the wonderful VAE-IAF codes !
I am trying to implement the codes with other datasets (time series data), but I have difficulties in running made.py file.
I guess the error occurs because of keras version.... etc.
Can you tell me which version of keras did you use in making made.py file ??
Thanks again !!

I don't understand what the N is in cls_loss in vae-m2-fit-mnist.ipynb in vae-semi_supervised_learning. Neither the original paper https://arxiv.org/pdf/1406.5298.pdf nor in your blog post it is specified.
If I'm not mistaken in your implementation this variable correspond to the number of labeled samples , however, I looked at the implementation in the original paper https://github.com/dpkingma/nips14-ssl/blob/master/learn_yz_x_ss.py (line 248) and they are using nb_samples / nb_labeled samples as N instead. Could you clear my confusion?

Thank you and keep up your good work! I'm finding your blog really helpful =)

I read your post on VAE. I am using pytorch resnet50 architecture as my encoder and have used a simple 5 layer transpose convolutions to get to the input dimension for decoder. The reconstructed image is noisy. There is not much change in the loss. I see that you have a VAE-resnet implementation, will that work for a huge data set with millions of images? I read another post of yours which talks about PCA trick. How is VAE resnet implemention results different from that of PCA trick implementation ? . I will also try with a pretrained resnet50 on imagenet to see whats the issue. Basically i am unable to rule out whether the problem is with the encoder, decoder or the dataset size. Any suggestion would help. Thanks.