input ref output result about reshapegan HOT 33 OPEN

@ak9250 I will also upload the code to merge all the synthesized images to a larger image, which is similar to the shown image in our paper. As you can see, the 1st, 3rd and 5th synthesized images have better reshaping performance than the 2nd and 4th outputs. The reason is that the geometric distance is too large between the input image and reference image. Our model fails to handle this task for the lack of prior knowledge. Even in the training stage, we could only obtain one image from one identity.

from reshapegan.

@zhengziqiang ok, also i suggest setting this up in google colab as most of these come preinstalled in there as well as scipy==1.2.0 is needed so you dont get imread or imsave errors

from reshapegan.

@ak9250 Ok, I will share my own codes to extract the facial landmarks for batch processing. But now it is too late, I will upload the codes tomorrow.

from reshapegan.

I am referring to these results https://github.com/zhengziqiang/ReshapeGAN#results

from reshapegan.

I have reorganized the codes, and you can generate one single image or multiple images at once.

from reshapegan.

thanks, there is no google drive like for celeba_images.zip in the Readme

from reshapegan.

@ak9250 I have uploaded the two files with the codes, you could download these with the added testing codes.

from reshapegan.

@zhengziqiang sorry, I see it now, also I tried it with my own images and the result demo_celeba.jpg is cut off and weird

from reshapegan.

same weird results with provided celeba test images

from reshapegan.

In our paper, we feed the model with the RGB image and geometric guidance (landmark, skeleton or even key point annotation). And for the weird results you mentioned, we have explained in our paper, if the distance between two face images is too large, the output would not be satisfactory. For example, if the input image is from a woman and the reference image is from a man, the generated image would be somehow weird. And you can run the test_multiple.py to generate more images and visualize all the results.

from reshapegan.

@ak9250 For the first case you mentioned, you should feed the model with real RGB image and geometric guidance.

from reshapegan.

@zhengziqiang can you share the script to get the geometric guidance for the test images?

from reshapegan.

You can also try other combinations by changing he path of input image or reference image. In theory, you could generate 1000*1000 images using 1000 images.

from reshapegan.

@zhengziqiang ok, can you please add the code on how you process the images to get the celeba_images test images with image and geometric guidance

from reshapegan.

Ok, I will upload the code to extract the geometric guidance as possible as soon. Any problem for the evaluation?

from reshapegan.

@zhengziqiang I just ran test multiple and here is the first result

from reshapegan.

To make it easier to show the reshaped results, I will upload the merge_results.py this evening.

from reshapegan.

@zhengziqiang thanks are you getting the 68 facial landmarks through the script, anything like dlib should work?

from reshapegan.

Yeah, I just used the dlib to obtain the facial landmark.

from reshapegan.

@ak9250 I used this code: https://github.com/davisking/dlib

from reshapegan.

@ak9250 Thank you for your suggestions, I have already uploaded the merge_results.py to make it easier to show the synthesized images.

from reshapegan.

@zhengziqiang thanks i tried out the merge results

I would like to try this with my own images but still setting up dlib for batch processing and combining, can you share your script as it would be faster?

from reshapegan.

@ak9250 If you have interesting on GAN, you could see my another article: EncryptGAN: Image Steganography with Domain Transform, to our knowledge, which is the first asymmetric image encryption system based on GAN. The paper could be found at https://arxiv.org/abs/1905.11582 and the code will be released at https://github.com/zhengziqiang/EncryptGAN

from reshapegan.

@zhengziqiang thanks, I ran extract_landmark.py and the celeba_land folder is empty

from reshapegan.

@zhengziqiang I was able to use another method and save the image does it need to be exactly in this format?

from reshapegan.

I was able to get it into that sort of format but results are weird still

from reshapegan.

@ak9250 You should feed the model the center cropped image. And the right of the input image should be the facial landmark image like this.

from reshapegan.

@zhengziqiang yes, I tried using extractor_image.py and it gives empty output, nothing in directory

from reshapegan.

@ak9250 You can try it now. The file list is empty while using glob.glob(data_path).

from reshapegan.

@zhengziqiang tried it now and it is working but still getting weird results

from reshapegan.

@zhengziqiang any idea why the weird results are happening?

from reshapegan.

@ak9250 You should normalize the both two images similar the testing data provided. Why do you use the input-reference images, which has different sizes of people? You only cropped and spatially normalized the reference image, and you don't implement the same operation to the input image. In our experiments, we have performed the same operation to the all the images, which could be found in facial_normalized.py. The two images have obviously different sizes (the facial landmarks)!

from reshapegan.

@zhengziqiang yes, I did use facial_normalized.py which gave me the merged and normalized results, I used the merged results in testing, will look into the normalized results

from reshapegan.