andrewgordonwilson / bayesgan Goto Github PK

In the testing set, basically what you are doing is first to find what images are not fake and then find the accuracy when they are classified as real.

So the accuracy can be really high when there is only a small number of images classified as real, which is kind of like cheating.

Hello,
I want to run the unsupervised BGAN with MNIST dataset using this command:
./run_bgan.py --data_path datasets --dataset mnist --num_mcmc 2 --out_dir ./results/ --train_iter 75000 --save_samples --n_save 100
But I got an error:

Starting session
Starting training loop
Iter 100
[None]
Gen losses = 3.56, 5.86
saving results and samples
Traceback (most recent call last):
  File "./run_bgan.py", line 296, in <module>
    b_dcgan(dataset, args)
  File "./run_bgan.py", line 103, in b_dcgan
    results = {"disc_losses": map(float, d_losses),
TypeError: float() argument must be a string or a number

I printed d_losses and got [None]. Any help?
Thank you

In lines 67-70:

if train_iter == 5000:
            print "Switching to user-specified optimizer"
            optimizer_dict = {"disc": dcgan.d_optims_adam,
                                          "gen": dcgan.g_optims_adam}

I believe what you intend to have here is dcgan.d_optims and dcgan.g_optims

I am struggeling to run your semi-supervised code on CIFAR. I have followed the README and set up tensorflow in conda accordingly with correct versions. When I run (as close as possible from the README):

./run_bgan_semi.py --data_path ./datasets/ --dataset cifar --num_gen 10 --num_mcmc 2 --out_dir cifar_out --train_iter 100000 --N 4000 --lr 0.0005

I get:

Iter 100
d_losses: [None]
disc_info: [None, None, 7.3618059, 7.3462958]
Traceback (most recent call last):
  File "./run_bgan_semi.py", line 419, in <module>
    b_dcgan(dataset, args)
  File "./run_bgan_semi.py", line 214, in b_dcgan
    print "Disc losses = %s" % (", ".join(["%.2f" % dl for dl in d_losses]))
TypeError: float argument required, not NoneType

This only happens at the 100th iteration (and I have printed the respective variables to show that there are unexpected Nones in there), so I guess the None types in the d_losses are not a problem before. Any ideas? Thanks for any help :).

Hello, I might be missing something obvious but I've been looking at it in every way possible and I can't grasp what I would be missing.

In the guide you submitted, a script, which is quite central to most of the things we can do with your repository, seems to be missing. It's the bayesian_gan_hmc script.

Do you know where I could find it? It would be of a great help to my problem.

Thank you.

Hello,
I was going through your paper and through the paper of SGHMC and I understand that :

• SGD + noise equates to SGLD
• SGD + momentum + noise equates to SGHMC
  However, I don't understand what Adam + noise which is what you used in your code equates to? in the paper you said that you were going to use SGHMC. Is it reasonable to assume that Adam + noise is also equivalent to SGHMC? if so, can you please say why?

Thanks

Hi Andrew,
I've just run bayesian_gan_hmc.py under tensorflow 1.3.0，here is the error:
Traceback (most recent call last): File "/home/jqh/jiangqiuhua/eclipse/plugins/org.python.pydev_6.2.0.201711281614/pysrc/pydevd.py", line 1621, in <module> main() File "/home/jqh/jiangqiuhua/eclipse/plugins/org.python.pydev_6.2.0.201711281614/pysrc/pydevd.py", line 1615, in main globals = debugger.run(setup['file'], None, None, is_module) File "/home/jqh/jiangqiuhua/eclipse/plugins/org.python.pydev_6.2.0.201711281614/pysrc/pydevd.py", line 1022, in run pydev_imports.execfile(file, globals, locals) # execute the script File "/home/jqh/jiangqiuhua/Tensorflow/bayesgan-master/bayesian_gan_hmc.py", line 442, in <module> b_dcgan(dataset, args) File "/home/jqh/jiangqiuhua/Tensorflow/bayesgan-master/bayesian_gan_hmc.py", line 141, in b_dcgan num_classes=dataset.num_classes if args.semi_supervised else 1) File "/home/jqh/jiangqiuhua/Tensorflow/bayesgan-master/bgan_models.py", line 357, in __init__ self.build_bgan_graph() File "/home/jqh/jiangqiuhua/Tensorflow/bayesgan-master/bgan_models.py", line 144, in build_bgan_graph self.generation["generators"].append(self.generator(self.z, gen_params)) File "/home/jqh/jiangqiuhua/Tensorflow/bayesgan-master/bgan_models.py", line 415, in generator h0 = tf.nn.relu(self.g_bn0(self.h0, reuse=reuse)) ValueError: Variable generator/g_bn0/moving_mean already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
Can I run this under tensorflow 1.3.0?

Dear authors, great work on the BayesGAN paper and code, congratulations! I had a question about your code - is there currently support for multiple discriminator MC samples (J_d > 1)? If not, is there any reason why it's left out? Thanks!

I am using tf1.3，it shows as follow, how to solve this problem?
Traceback (most recent call last):
File "I:/python3/bayesgan-master/run_bgan_semi.py", line 413, in
b_dcgan(dataset, args)
File "I:/python3/bayesgan-master/run_bgan_semi.py", line 141, in b_dcgan
num_classes=dataset.num_classes)
File "I:\python3\bayesgan-master\bgan_semi.py", line 78, in init
self.build_bgan_graph()
File "I:\python3\bayesgan-master\bgan_semi.py", line 274, in build_bgan_graph
self.K, disc_params)
File "I:\python3\bayesgan-master\bgan_semi.py", line 401, in discriminator
w=disc_params["d_h%i_W" % layer], biases=disc_params["d_h%i_b" % layer]), train=train))
File "I:\python3\bayesgan-master\dcgan_ops.py", line 46, in call
scope=self.name)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\framework\python\ops\arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\layers\python\layers\layers.py", line 592, in batch_norm
scope=scope)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\layers\python\layers\layers.py", line 373, in _fused_batch_norm
collections=moving_mean_collections)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\framework\python\ops\arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\framework\python\ops\variables.py", line 262, in model_variable
use_resource=use_resource)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\framework\python\ops\arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\contrib\framework\python\ops\variables.py", line 217, in variable
use_resource=use_resource)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1203, in get_variable
constraint=constraint)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1092, in get_variable
constraint=constraint)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 425, in get_variable
constraint=constraint)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 394, in _true_getter
use_resource=use_resource, constraint=constraint)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 742, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable discriminator/d_bn1/moving_mean already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 2956, in create_op
op_def=op_def)
File "D:\ProgramData\tensorflow\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)

I'm just wondering, shouldn't we use independent batch_norm variables for each generation of the generator ?

To learn on custom data (not images), are the shapes hard-coded to 8x8 or I can change them? I essentially have flat vectors.

Hi Andrew,
I've just done some experiments with WGAN with Gradient Penalty (Improved Training of Wasserstein GANs, Gulrajani et al.) and found that it can converge to a reasonable solution on the synthetic dataset. Although WGANGP does not converge as fast as bayesgan, I think it would be nice if you could add WGANGP to the baselines in your experiments.
Here is the output of my (very bad) implementation of WGANGP after 8000 iterations

nx = 100
nz = 10
batchSize = 64
Gconfig = [('Linear', (nz, 1000)), ('ReLU', ()), ('Linear', (1000, nx))]
Dconfig = [('Linear', (nx, 1000)), ('ReLU', ()), ('Linear', (1000, 1))]
optimizer = 'Adam'
optimParams = {'lr': 1e-4, 'betas': (0.5, 0.9)}

In attempting run the MNIST example (with the below, most basic command) I ran into the below error.

Traceback (most recent call last):
  File "bayesgan/bayesian_gan_hmc.py", line 431, in <module>
    b_dcgan(dataset, args)
  File "bayesgan/bayesian_gan_hmc.py", line 158, in b_dcgan
    optimizer_dict = {"semi_d": dcgan.d_optim_semi_adam,
AttributeError: 'BDCGAN' object has no attribute 'd_optim_semi_adam'

My environment is compiled with the environment.yaml and my command to run the code is:

bayesgan/bayesian_gan_hmc.py --data_path /home/jlandesman --dataset mnist --out_dir results/mnist --save_samples --n_save 100

The synth data appears to work well.

Any thoughts?

Many thanks for your help and congratulations on the paper.

Dear authors,
As can be seen from generated samples in figure 2, 6, 7 and 8 mode collapse is a serious problem in Bayesian GAN. Every generator has mode collapse and different generators collapse to the same modes.
In figure 6, for example, generator 1 and 4 both have mode collapse and they collapse to the same mode (row 2, col 3 of generator 1 and row 3, col 3 of generator 4). If we consider the mode count method based on birthday paradox (Arora et al. 2017) then when mode collapse happens with high probability then the number of mode in the model distribution is about the same as the batch size. Mode collapse happens with batch size of only 16, that implies that each generator captures only tens of modes. The total capacity of 10 generators is, therefore, much smaller than a single generator trained with normal method.
This is contrast to your claim that Bayesian GAN explore a broader region of the target distribution. In my opinion, the current setting for Bayesian GAN makes mode collapse worse.

If I have generated numpy matrices as my "real data", do I specify that that as 'self.imgs = np.load('matrices.npy')'?
I intend to run this in unsupervised mode, so do I need to supply the other parameters such as self.labels, self.test_imgs and self.test_labels?
Sorry if this is a basic question, I am fairly new to GANs.