shekkizh / wassersteingan.tensorflow Goto Github PK

It's really a great work for the implementation of WGAN. This code is very clear, readable and well-written. However, when I try to run the code under TF v0.12.1, I got an error

Traceback (most recent call last):
File "main.py", line 52, in
tf.app.run()
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 43, in main
model.initialize_network(FLAGS.logs_dir)
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 227, in initialize_network
self.sess.run(init)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]

Caused by op u'Placeholder', defined at:
File "main.py", line 52, in
tf.app.run()
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 41, in main
FLAGS.optimizer_param)
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 173, in create_network
self._setup_placeholder()
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 149, in _setup_placeholder
self.train_phase = tf.placeholder(tf.bool)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1512, in placeholder
name=name)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
name=name)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]

I try to fix it. But I don't have an idea where I should get started.

I test

import tensorflow as tf
a = tf.placeholder(tf.bool)
b = tf.constant(2)
c = tf.constant(3)
d = tf.cond(a, lambda: tf.add(b, c), lambda: tf.mul(b, c))
init  = tf.global_variables_initializer()
tf.Session().run(init)
tf.Session().run(d, {a: True})

It runs correctly.

So, would you mind giving a hint where should I start to debug?

Thank you very much.

I have a question on the discriminator construction. I find the final number of channel is "1" via convolutional layer in this implementation. However, I find in others, e.g., "improved wgan", the final layer is fully-connection layer with the out dimension "1".
So, which one is better? Indeed, I do not find any description of discriminator construction in the original paper (Wasserstein GAN).

according to the wgan paper , when training the critic, we need to maximize Er(f(x)) - Eg(f(x)), and when training the generator, we need to minimize Er(f(x)) - Eg(f(x)), that is to say minimize - Eg(f(x)), but in the code, wgan's loss use self.discriminator_loss = tf.reduce_mean(logits_real - logits_fake) , and self.gen_loss = tf.reduce_mean(logits_fake) , so I'm confused.

Excellent implement!!!
I want to implement WGAN in Caffe. I have confused with the gradient computing of WGAN loss.
Would you give some details of mathematical formulas?
Thank you!

I think your WGAN loss if not right, it should be:
self.discriminator_loss = tf.reduce_mean(logits_fake - logits_real)
self.gen_loss = tf.reduce_mean(-logits_fake)
ans your WGAN discriminator output shape is [batch_size, channel=1, height=2, width=2],it is a mistake?

Hi. In this implementation, is batch normalization used in the discriminator? I think in WGAN paper, it is mentioned that you should not use BN in discriminators, right?

In discriminators it seems that the biases term in all convolutional layers are set fixed, and to zero. I don't think this is mentioned in the paper either.

The tensorflow version I use is 0.12.0.
I run the main.py with: python main.py --logs_dir=logs/CelebA_WGAN_logs2/ --optimizer=RMSProp --learning_rate=5e-5 --optimizer_param=0.9 --model=1 --iterations=1e5 --mode=visualize
It shows the following error:

Traceback (most recent call last):
File "main.py", line 52, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 43, in main
model.initialize_network(FLAGS.logs_dir)
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 225, in initialize_network
self.sess.run(tf.global_variables_initializer())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]

Caused by op u'Placeholder', defined at:
File "main.py", line 52, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 41, in main
FLAGS.optimizer_param)
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 173, in create_network
self._setup_placeholder()
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 149, in _setup_placeholder
self.train_phase = tf.placeholder(tf.bool)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1587, in placeholder
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]

Does anyone has any idea about it? Much thanks!

Hi,
Much thanks to your excellent work!
BTW, I am a little bit confused with one line code, which in GAN_models.py line 335: "self.discriminator_loss = tf.reduce_mean(logits_real - logits_fake)" . Shouldn't it be "self.discriminator_loss = tf.reduce_mean(logits_fake - logits_real)"? According to equation (3) in the original paper(Wasserstein GAN,Martin Arjovsky,p.7), it seems that the discriminator_loss should be maximized.

Thank you for sharing the code.

Say if,
D should maximise D(f)-D(r) , => minimising D(r)- D(f)
and G should minimise D(f)-D(r) => minimising D(f).

And your code also follows the same logic. But I'm unable to comprehend from D loss curve, why is the loss increasing from ( -15 to around zero). Shouldn't the curve go down as iterations happen, since we are minimising the loss( D(r)-D(f)?

Please share your thoughts

I don't understand how exactly the loss function in line 5 of algorithm 1 in the original WGAN paper is implemented here. In your code you minimise

self.discriminator_loss = discriminator_loss_fake + discriminator_loss_real

However, according to the paper shouldn't it be maximising:

self.discriminator_loss = discriminator_loss_real - discriminator_loss_fake

or alternatively minimising:

self.discriminator_loss = discriminator_loss_fake - discriminator_loss_real

That is, should this be a minus in your total loss?

My tensorflow version is 0.12.1

when I run run_main.py, I got this error

"ValueError: Variable discriminator/disc_bn1/discriminator_1/disc_bn1/cond/discriminator_1/disc_bn1/moments/moments_1/mean/ExponentialMovingAverage/biased does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?"

Any one has any idea?

Dear Author

Thanks a lot for providing the code of W-GAN. BTW, we want to know if there is any pre-trained W-GAN model for us to directly test without training? Since our device is now allowed to train such a big model. Thanks a lot.

Regards,
Vic