when I run the commond of "./run_convert_market.sh" in TF-record data preparation steps and the result is as follows:
100000
200000
repeat positive pairs augment_ratio times and cut down negative pairs to balance data ......
p_pairs length:296938
n_pairs length:41936
save p_pairs and n_pairs ......
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 911, in run_one_pair_rec
mode='same_diff_cam')
File "convert_market.py", line 202, in _get_train_all_pn_pairs
pickle.dump(p_pairs,f)
TypeError: write() argument must be str, not bytes

You said you used a pose estimator to generate the target pose and transformed them into 18 heatmaps. Is that procedure included in this code? Or the dataset you offered already contains the target pose?

Hi!

I can't download your dataset because of forbidden error. Can you please restore an access to it?
Thanks in advance.

Hi,

Thanks for your quickly update code.

I found that there might exist some errors in run_DF_test.sh and run_DF_train.sh.

if [ ! -d ./data/DF_train_data ]; then
    cd data
    wget homes.esat.kuleuven.be/~liqianma/NIPS17_PG2/data/DF_train_data.zip
    unzip DF_train_data.zip
    rm -f DF_train_data.zip
    cd -
fi

run_DF_test.sh and run_DF_train.sh check /data/DF_train_data and /data/DF_test_data respectively. If the folders they wanted do not exist, they will download DF_train_data.zip and DF_test_data.zip. However, I checked the DF_train_data.zip and DF_test_data.zip which I downloaded several hours ago - I do not sure if you updated those files recently. I found that the the folder structures in DF_train_data.zip and DF_test.zip are totally different to the run_DF_train.sh and run_DF_test.sh

Archive:  DF_train_data.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-09-29 15:58   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/
  2657209  2017-09-29 02:20   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/n_pairs_train.p
123995280280  2017-09-29 11:37   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/DeepFashion_train_00000-of-00001.tfrecord
        9  2017-09-29 02:20   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/pn_pairs_num_train.p
  3064685  2017-09-29 02:20   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/p_pairs_train.p
        9  2017-09-29 11:37   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_sparse_partBbox37_maskR4R8_roi10Complete/tf_record_pair_num.txt
---------                     -------
124001002192                     6 files

Archive:  DF_test_data.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-11-09 21:44   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/
21644244963  2017-11-09 21:37   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/DeepFashion_test_00000-of-00001.tfrecord
   726567  2017-11-09 19:48   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/p_pairs_test.p
   339187  2017-11-09 19:48   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/n_pairs_test.p
        8  2017-11-09 19:48   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/pn_pairs_num_test.p
        9  2017-11-09 21:37   data4tf_GAN_attr_pose_onlyPosPair_256x256PoseRCV_Mask_test_sparse_partBbox37_maskR4R8_roi10Complete/tf_record_pair_num.txt
---------                     -------
21645310734                     6 files

I have trained the model and want to use that for testing which ckpt file am i supposed to use for testing as i think the run_DF_test file was meant for pretrained model

I am trying to train the model with another dataset Lookbook. The inputs are image_raw, target_image, pose and mask, the size of target pose is (256,256,9). I got the following error when start training. It happened at this line in function get_image_from_loader(self) (i modified your code):

x, x_target, pose_target, mask_target = self.sess.run([self.x, self.x_target,self.pose_target, self.mask_target])

OutOfRangeError (see above for traceback): FIFOQueue '_4_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /home/ychen/git/Pose-Guided-Person-Image-Generation/train_lookbook.py:114) = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]

I am quite new to Tensorflow, any help is much appreciated!

OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 32, current size 0) [[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]] [[Node: batch/_1515 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_1 0_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

I select several images from test clean set (not in test sample set), and convert the images by your scripts. But I met the above bug during the inference. Could you give me any suggestions?

What is the intuition behind using the triplet scheme as compared to the pair wise scheme used in the paper? What kind of improvements does it lead to?

Even after lots of debugging, the downloaded pre-trained model cannot be loaded with the following error:

DataLossError (see above for traceback): Unable to open table file ./pretrain_models/PG2_model_D│face_morpher Sketch-Photo-Conversion-using-Deep-CNN
F/model.ckpt-0.data-00000-of-00001: Data loss: not an sstable (bad magic number): perhaps your f│FastPhotoStyle SketchToFace
ile is in a different file format and you need to use a different restore operator?

"You don't have permission to access ..."
Would you please to fix it?

Hi,

I could not access any of the datasets / trained models. The server is returning ERROR 403: Forbidden. Will you please fix the problem again? Thanks!

I'm aware this has happened before and was addressed. The issue is back again.

There is an error when I run run_DF_test.sh.

AttributeError: 'PG2_256' object has no attribute 'G'

I think we have to assign self.G as self.G2 in the build_model model of PG2_256 in trainer256.py:

    self.G1 = denorm_img(G1, self.data_format)
    self.G2 = denorm_img(G2, self.data_format)
    self.G = self.G2   # Add this to fix no attribute 'G' error

How to use pretrained model for single image pose change

Dataset and tf_records links are broken

According to your code, generating 18 keypoints depends on file——annotation - {type}. csv,
so how to generate my own data set's xxxx.csv file?

Hi, I am porting PG2 to pytorch.
I am confused about the design of Discriminator:
In your code, the last layer of Discriminator is a linear whose output maybe not in [0, 1].

so, I add a sigmoid to output:

        self.fc = nn.Linear(8*4*8*base_channels, 1)
        self.sigmoid = nn.Sigmoid()

        self.main.apply(weights_init)

    def forward(self, x):
        x = self.main(x)
        x = x.view(-1, 8*4*8*self.base_channels)
        return self.sigmoid(self.fc(x))

copy from here

This leads to poor output

if I only change the input layer's stride to (4, 2) of DCGAN's Discriminator.

            nn.Conv2d(in_channels=nc, out_channels=nf, kernel_size=4, stride=(4, 2), padding=1, bias=False),
            nn.LeakyReLU(0.2, inplace=True),

I get a better output.

but it's still far from your result in paper.

Hi,
I am unable to identify the function for pose estimation. I believe that the tf-records of test data already contain precomputed pose information (Correct me if that is not the case). Could you please direct me to the pose estimation and mask estimation function?

Thanks

the datasets folder is referred as data in the run_DF_test.sh script.
in the config.py, the default datasets folder is set to 'data'.

When running the script, it complains that the 'data' folder can not be found.

My computer Dependencies:

python 2.7
tensorflow-gpu (1.4.0)
numpy (1.15.1)
Pillow (5.2.0)
scikit-image (0.14.0)
scipy (0.17.0)
matplotlib (2.2.3)

After I run run_DF_train.sh. I appeared this problem:IndexError: list index out of range.The picture is here:

Looking forward to your reply,Thank you.

Hi. When i tried to process data with "run_convert_DF.sh" i get this error. Do you know how i can solve it?
Thanks

File "convert_DF.py", line 1005, in <module>
    run_one_pair_rec(dataset_dir, out_dir, split_name)
  File "convert_DF.py", line 982, in run_one_pair_rec
    mode='same_diff_cam')
  File "convert_DF.py", line 172, in _get_train_all_pn_pairs
    pickle.dump(p_pairs,f)
TypeError: write() argument must be str, not bytes

@charliememory Thanks for your patience. (￣▽￣)", I have implemented your paper with Pytorch.

Pytorch implemention: Human-Pose-Transfer
result:

https://github.com/budui/Human-Pose-Transfer#implement-result
In addition, I also implemented PatchGAN-PG2, LSLoss-PG2

target PG2 LSLoss-PG2 PatchGAN-PG2

What is the no. of epochs and hidden layer in this project?

When I use the downloaded test data set and test with the trained model, the generated image is not the same as the target pose.

Hi~When I want to run the run_DF_train.sh,I got a OutOfRangeError:

OutOfRangeError (see above for traceback): FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
 [[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, b batch/n)]]

,Some TraceBacks:

tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_3_batch/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, batch/n)]]

Caused by op u'batch', defined at:
  File "main.py", line 37, in <module>
    main(config)
  File "main.py", line 24, in main
    trainer = PG2_256(config)
  File "/Pose-Guided-Person-Image-Generation-master/trainer256.py", line 19, in __init__
    self.x, self.x_target, self.pose, self.pose_target, self.mask, self.mask_target = self._load_batch_pair_pose(self.dataset_obj)

I do exactly as your training steps,Can you give me some advice to fix it?
Thanks~

I would like to know what are the required system resources (i.e. CPU, GPU, RAM), in order to train and test this model?

Since your datasets format is .p, I do not know how to make the training datasets with my image, Can you talk about the detail (like parameters)of the datasets format?
Thank you very much.

I have made changes in the lines:
By replacing o._shape = tf.TensorShape(new_shape) with o.set_shape (tf.TensorShape(new_shape))
and logits = tf.matmul(tf.squeeze(pool3), w) with logits = tf.matmul(tf.squeeze(pool3,[1,2]), w) to remove the previous errors but Now I am facing the below value error:

Traceback (most recent call last):
File "score_mask.py", line 107, in
IS_G_mean, IS_G_std = tflib.inception_score.get_inception_score(G_list_masked)
File "/mocapdata/Students/sheela/Pose-Guided-Person-Image-Generation/tflib/inception_score.py", line 46, in get_inception_score
pred = sess.run(softmax, {'ExpandDims:0': inp})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 930, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1129, in _run
str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (100, 256, 256, 3) for Tensor u'ExpandDims:0', which has shape '(1, ?, ?, 3)'

It would be really helpful if someone can give me the solution?

In the market1501 dataset, I am confused about the meaning of bounding_box_test_clean.

Thanks for your time.

Hi~I get a 403 Forbiddien when I want to download the training data in tf-reocrd format.And it looks like all the resources can't be downloaded.Can you please fix it?

Hi,

I've used these settings that you specify in the run_DF_train.sh and could not reproduce the results as provided in your pretrained model link DeepFashion. The results from testing your pretrained model(model.ckpt-0) look much better than what I got from training(model.ckpt-77999) and using it for testing.

gpu=0
D_arch='DCGAN'
stage=1

model_dir="./models/train"

## Make sure dataset name appear in  --dataset  (i.e. 'Market' or 'DF')
python main.py --dataset=DF_train_data \
             --img_H=256  --img_W=256 \
             --batch_size=1 --max_step=80000 \
             --d_lr=0.00002  --g_lr=0.00002 \
             --lr_update_step=50000 \
             --is_train=True \
             --model=11 \
             --D_arch=${D_arch} \
             --gpu=${gpu} \
             --z_num=64 \
             --model_dir=${model_dir} \

Hi! Thanks for your open-source code of your project. I have few questions about some details in trainer.py.
Why do you cycle 400 times in the test procedure?
Is every iteration better than the previous one?
Thanks a lot!

Would you explain the steps involved in dataset preparation including the data pairing for training dataset. I want to train my own data. So it would be helpful. Looking for yours positive response.

Why has mask_target not been used for stage 1? The paper mentions it's use there. I tried to train a model without mask_target in stage 1 and the results have not been as good as the ones shown in the paper.

When I load the pre-trained DeepFusion model, it encounters shape mismatch.

Pretrained model: UAEnoFC/G/conv18: [3, 3, 256, 128]
current sesson graph: UAEnoFC/G/conv18: [3, 3, 128, 128]

Could you please clarify the graph definition in trainer.py?

According to the article, The pose estimator generates the coordinates of 18 keypoints, then you encode pose as 18 heatmaps
but I can't find the function which converts the coordinates to heatmaps.
can you help me, please?

I can't find the supplementary material mentioned in the paper,could you give me give a link, thank you!

Now I want to test the code with other datasets,
But I don't know how Market1501_test_00000-of-00001.tfrecord, n_pairs_test.p, p_pairs_test.p, pn_pairs_num_test.p are produced in the Market_test_data.zip folder.

Hi,

Thank you for your code. :)
May I know the intuition behind no batch normalization in your work?

Hi, I'm trying to download the resources but I got :
You don't have permission to access /~liqianma/NIPS17_PG2/models/DF_model.zip on this server.

Could you please fix the problem?
Thanks!

When i tried to run testing with your provide pretrained model. I get this error

Traceback (most recent call last):
  File "score_mask.py", line 167, in <module>
    masked_G1_array = np.uint8(mask_target_list[i][:,:,np.newaxis]/255.*G1_list[i])
IndexError: list index out of range

Do you know how to solve this problem?
Thanks

I want to download the resource such as datasets,etc to complete my experiments. But there is a 403 forbidden error. Can you fix it? it might be helpful. Thanks!

If I want to use other image to test the model performance, how would I do?
Thank you very much

@charliememory Once I have trained the model, I want to run Pose Transfer on a given image. But the current implementation required a tfrecord file to be generated from images and poses, and applies pose transfer on those. Any quick solution for that ?

Download links on your "Resources" are not work!

Please fix it~~

@charliememory I am trying to freeze the model from pbtxt and ckpt files but can not able to find output and input node name. Can you please tell how to freeze the model graph and use it to convert to tensorflow lite model. Also if you have frozen graph can you please share ?

Hi, Liqian,
Thanks for your open-source code of your project. I have few questions about some details in your network:
Q1: In [1]:
I note that in the decoders of G1 and G2, you use upscale + conv to up-sample the feature map, and you comment a line in which transposed conv is used to up-sample. These two methods, do they have big influence on the final performance?
Just see code here

Q2:Based on Q1, which of method you used in your paper [2]?

Q3: In [2]:
In Sec 3.4, what the expressions "fully-connected residual layers" mean?

Because your illustrations in supplementary materials looks that the "mapping functions" and "Pose auto-encoder" just consists of simple fully-connections. I haven't found any "residual" elements.

Looking forward to your reply.

References:
[1]Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., & Van Gool, L.. Pose Guided Person Image Generation. NIPS2017
[2]Disentangled Person Image Generation