shekkizh / wassersteingan.tensorflow Goto Github PK
View Code? Open in Web Editor NEWTensorflow implementation of Wasserstein GAN - arxiv: https://arxiv.org/abs/1701.07875
License: MIT License
Tensorflow implementation of Wasserstein GAN - arxiv: https://arxiv.org/abs/1701.07875
License: MIT License
according to the wgan paper , when training the critic, we need to maximize Er(f(x)) - Eg(f(x)), and when training the generator, we need to minimize Er(f(x)) - Eg(f(x)), that is to say minimize - Eg(f(x)), but in the code, wgan's loss use self.discriminator_loss = tf.reduce_mean(logits_real - logits_fake) , and self.gen_loss = tf.reduce_mean(logits_fake) , so I'm confused.
Thank you for sharing the code.
Say if,
D should maximise D(f)-D(r) , => minimising D(r)- D(f)
and G should minimise D(f)-D(r) => minimising D(f).
And your code also follows the same logic. But I'm unable to comprehend from D loss curve, why is the loss increasing from ( -15 to around zero). Shouldn't the curve go down as iterations happen, since we are minimising the loss( D(r)-D(f)?
Please share your thoughts
Dear Author
Thanks a lot for providing the code of W-GAN. BTW, we want to know if there is any pre-trained W-GAN model for us to directly test without training? Since our device is now allowed to train such a big model. Thanks a lot.
Regards,
Vic
Hi. In this implementation, is batch normalization used in the discriminator? I think in WGAN paper, it is mentioned that you should not use BN in discriminators, right?
In discriminators it seems that the biases term in all convolutional layers are set fixed, and to zero. I don't think this is mentioned in the paper either.
I think your WGAN loss if not right, it should be:
self.discriminator_loss = tf.reduce_mean(logits_fake - logits_real)
self.gen_loss = tf.reduce_mean(-logits_fake)
ans your WGAN discriminator output shape is [batch_size, channel=1, height=2, width=2],it is a mistake?
The tensorflow version I use is 0.12.0.
I run the main.py with: python main.py --logs_dir=logs/CelebA_WGAN_logs2/ --optimizer=RMSProp --learning_rate=5e-5 --optimizer_param=0.9 --model=1 --iterations=1e5 --mode=visualize
It shows the following error:
Traceback (most recent call last):
File "main.py", line 52, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 43, in main
model.initialize_network(FLAGS.logs_dir)
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 225, in initialize_network
self.sess.run(tf.global_variables_initializer())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]
Caused by op u'Placeholder', defined at:
File "main.py", line 52, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 41, in main
FLAGS.optimizer_param)
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 173, in create_network
self._setup_placeholder()
File "/home/xujingwei/WassersteinGAN.tensorflow/models/GAN_models.py", line 149, in _setup_placeholder
self.train_phase = tf.placeholder(tf.bool)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1587, in placeholder
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]
Does anyone has any idea about it? Much thanks!
My tensorflow version is 0.12.1
when I run run_main.py, I got this error
"ValueError: Variable discriminator/disc_bn1/discriminator_1/disc_bn1/cond/discriminator_1/disc_bn1/moments/moments_1/mean/ExponentialMovingAverage/biased does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?"
Any one has any idea?
Excellent implement!!!
I want to implement WGAN in Caffe. I have confused with the gradient computing of WGAN loss.
Would you give some details of mathematical formulas?
Thank you!
I don't understand how exactly the loss function in line 5 of algorithm 1 in the original WGAN paper is implemented here. In your code you minimise
self.discriminator_loss = discriminator_loss_fake + discriminator_loss_real
However, according to the paper shouldn't it be maximising:
self.discriminator_loss = discriminator_loss_real - discriminator_loss_fake
or alternatively minimising:
self.discriminator_loss = discriminator_loss_fake - discriminator_loss_real
That is, should this be a minus in your total loss?
I have a question on the discriminator construction. I find the final number of channel is "1" via convolutional layer in this implementation. However, I find in others, e.g., "improved wgan", the final layer is fully-connection layer with the out dimension "1".
So, which one is better? Indeed, I do not find any description of discriminator construction in the original paper (Wasserstein GAN).
Hi,
Much thanks to your excellent work!
BTW, I am a little bit confused with one line code, which in GAN_models.py line 335: "self.discriminator_loss = tf.reduce_mean(logits_real - logits_fake)" . Shouldn't it be "self.discriminator_loss = tf.reduce_mean(logits_fake - logits_real)"? According to equation (3) in the original paper(Wasserstein GAN,Martin Arjovsky,p.7), it seems that the discriminator_loss should be maximized.
It's really a great work for the implementation of WGAN. This code is very clear, readable and well-written. However, when I try to run the code under TF v0.12.1, I got an error
Traceback (most recent call last):
File "main.py", line 52, in
tf.app.run()
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 43, in main
model.initialize_network(FLAGS.logs_dir)
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 227, in initialize_network
self.sess.run(init)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]Caused by op u'Placeholder', defined at:
File "main.py", line 52, in
tf.app.run()
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "main.py", line 41, in main
FLAGS.optimizer_param)
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 173, in create_network
self._setup_placeholder()
File "/home/qiqi/code/wgan/WassersteinGAN.tensorflow/models/GAN_models.py", line 149, in _setup_placeholder
self.train_phase = tf.placeholder(tf.bool)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1512, in placeholder
name=name)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
name=name)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/qiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype bool
[[Node: Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]]
I try to fix it. But I don't have an idea where I should get started.
I test
import tensorflow as tf
a = tf.placeholder(tf.bool)
b = tf.constant(2)
c = tf.constant(3)
d = tf.cond(a, lambda: tf.add(b, c), lambda: tf.mul(b, c))
init = tf.global_variables_initializer()
tf.Session().run(init)
tf.Session().run(d, {a: True})
It runs correctly.
So, would you mind giving a hint where should I start to debug?
Thank you very much.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.