Recently, realistic image generation using deep neural networks has become a hot topic in machine learning and computer vision. Such an image can be generated at pixel level by learning from a large collection of images. Learning to generate colorful cartoon images from black-and-white sketches is not only an interesting research problem, but also a useful application in digital entertainment. In this paper, we investigate the sketch-to-image synthesis problem by using conditional generative adversarial networks (cGAN). We propose a model called auto-painter which can automatically generate compatible colors given a sketch. Wasserstein distance is used in training cGAN to overcome model collapse and enable the model converged much better. The new model is not only capable of painting hand-draw sketch with compatible colors, but also allowing users to indicate preferred colors. Experimental results on different sketch datasets show that the auto-painter performs better than other existing image-to-image methods.
I downloaded VGG16 model from https://github.com/machrisaa/tensorflow-vgg and put it in my_vgg folder. However, when I run the program, it shows the log like this. Could you help me with this error? Thank you.
/usr/bin/python2.7 /data2/texttophoto/Auto_painter/training&test/auto-painter.py --mode train --input_dir /data2/texttophoto/Auto_painter/preprocessing/gen_sketch/pic_sketch/ --output_dir /data2/texttophoto/Auto_painter/preprocessing/gen_sketch/pic_color/ --checkpoint None
aspect_ratio = 1.0
batch_size = 1
beta1 = 0.5
checkpoint = None
display_freq = 0
f_weight = 0.0001
flip = True
gan_weight = 1
input_dir = /data2/texttophoto/Auto_painter/preprocessing/gen_sketch/pic_sketch/
l1_weight = 50.0
lr = 2e-05
max_epochs = 20
max_steps = None
mode = train
ndf = 48
ngf = 48
output_dir = /data2/texttophoto/Auto_painter/preprocessing/gen_sketch/pic_color/
output_filetype = png
progress_freq = 50
save_freq = 5000
scale_size = 530
seed = 1242561248
summary_freq = 100
trace_freq = 0
tv_weight = 0.001
examples count = 819
/data2/texttophoto/Auto_painter/training&test/my_vgg/vgg16.npy
npy file loaded
/data2/texttophoto/Auto_painter/training&test/my_vgg/vgg16.npy
npy file loaded
Traceback (most recent call last):
File "/data2/texttophoto/Auto_painter/training&test/auto-painter.py", line 714, in
main()
File "/data2/texttophoto/Auto_painter/training&test/auto-painter.py", line 549, in main
model = create_model(examples.inputs, examples.targets, net1, net2)
File "/data2/texttophoto/Auto_painter/training&test/auto-painter.py", line 373, in create_model
gen_loss_f = tf.reduce_mean(tf.abs(feature_loss(targets,net1) - feature_loss(outputs,net2)))
File "/data2/texttophoto/Auto_painter/training&test/auto-painter.py", line 335, in feature_loss
vgg.build(image)
File "/data2/texttophoto/Auto_painter/training&test/my_vgg/vgg16.py", line 36, in build
assert red.get_shape().as_list()[1:] == [224, 224, 1]
AssertionError
build model started
Process finished with exit code 1
import numpy as np
import random
res=np.zeros([1000,100])
for i in range(1000):
for j in range(100):
p=random.random()
w=random.random()/2.5
if p <w:
res[i,j] = 0
else:
res[i, j] = 1
i have tried two ways in training, one is training directly on the datasets, another is first using pre-trained model offered by the author then training on the datasets. both of them are tested out all green outputs as follows, one is input picture, another is output. can anybody tell me why. thanks a lot.
When I tried loading the pre-trained model, something went wrong like below. It seems that the model's parameters doesn't match the structure?
Can anyone help? Thank you very much!
2020-01-04 20:49:38.604178: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam_1 not found in checkpoint
2020-01-04 20:49:38.604178: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/scale/Adam not found in checkpoint
2020-01-04 20:49:38.604221: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/conv/filter/Adam not found in checkpoint
2020-01-04 20:49:38.604253: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/scale/Adam_1 not found in checkpoint
2020-01-04 20:49:38.604195: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam not found in checkpoint
2020-01-04 20:49:38.606036: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_8/batchnorm/offset/Adam_1 not found in checkpoint
2020-01-04 20:49:38.606443: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/conv/filter/Adam_1 not found in checkpoint
2020-01-04 20:49:38.606492: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_8/batchnorm/offset/Adam not found in checkpoint
2020-01-04 20:49:38.607934: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_8/batchnorm/scale/Adam not found in checkpoint
2020-01-04 20:49:38.612170: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam not found in checkpoint
[[Node: save/RestoreV2_151 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_151/tensor_names, save/RestoreV2_151/shape_and_slices)]]
2020-01-04 20:49:38.612178: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam not found in checkpoint
[[Node: save/RestoreV2_151 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_151/tensor_names, save/RestoreV2_151/shape_and_slices)]]
2020-01-04 20:49:38.627436: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam not found in checkpoint
[[Node: save/RestoreV2_151 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_151/tensor_names, save/RestoreV2_151/shape_and_slices)]]
2020-01-04 20:49:38.627436: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Key generator/encoder_7/batchnorm/offset/Adam not found in checkpoint
[[Node: save/RestoreV2_151 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_151/tensor_names, save/RestoreV2_151/shape_and_slices)]]
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent