ReshapeGAN is a Tensorflow-based framework for training and testing of our paper ReshapeGAN: Object Reshaping by Providing A Single Reference Image.
We updated this version and achieve the high-resolution image-to-image translation on CelebA-HQ and FFHQ dataset for both 1024*1024 and 512*512. Besides, we also add the triplet loss to better preserve the identity information. And we will release the pre-trained model soon. here are some visual results.
- We use Miniconda3 as the basic environment. If you have installed the Miniconda3 in path
Conda_Path, please installtensorflow-gpuusing the commandConda_Path/bin/conda install -c anaconda tensorflow-gpu==1.8. - Install dependencies by
Conda_Path/bin/pip install -r requirements.txt(if necessary). Therequirements.txtfile is provided in this package.
The training code will be released soon!
We provide the pre-trained model and 1000 testing images for evaluation.
Pre-trained model celeba.zip (Google Drive and Baidu Drive).
Testing images used in our paper: celeba_images.zip.
After download the pre-trained model and testing images, run
mkdir checkpoint
mv celeba.zip checkpoint/
cd checkpoint/
unzip celeba.zip
cd ..
mkdir -p data/celeba
mv celeba_images.zip data/celeba/
cd data/celeba
unzip celeba_images.zip
cd ../..
Then we provide two files: test_single.py and test_multiple.py for generating only one image and multiple images respectively. For generating one image:
Conda_path/bin/python test_single.py # generating one image at once
--input_img ./data/celeba/celeba_images/0000.jpg # input image, you could also change this to any input image (0001.jpg, 0002.jpg and etc)
--ref_img ./data/celeba/celeba_images/0001.jpg # reference image, you could also change this to any reference image (0001.jpg, 0002.jpg and etc)
--checkpoint_dir ./checkpoint/celeba # checkpoint path
--result_img ./demo_celeba.jpg # result image
For generating multiple images using different reference images:
Conda_path/bin/python generate_multiple.py # generate combinations of reference images, one combination has 5 different reference images, and we randomly use one image as input image.
--data_path ./data/celeba/celeba_images # images path for providing reference images
--save_path ./data/celeba/combination # path to save image combinations
--n_ref # how many reference images in one image combination, we set default is 5
Conda_path/bin/python test_multiple.py
--input_dir ./data/celeba/combination # input image path
--checkpoint_dir ./checkpoint/celeba # checkpoint path
--result_dir ./result/celeba # result path
To make it easier to show the synthesized images, we also provide the merge_result.py to merge the generated images to a larger image.
Conda_path/bin/python merge_results.py
--data_path ./result/celeba # the results path
--save_path ./result/celeba/merge # the path to save merged images.
We merge 96 synthesized images to a large image, and the middle enlarged image is the input image.
We provide the pre-trained model and 1000 testing images for evaluation.
Pre-trained model utk.zip (Google Drive and Baidu Drive).
Testing images used in our paper: utk_images.zip.
After download the pre-trained model and testing images, run
mv utk.zip checkpoint/
cd checkpoint/
unzip utk.zip
cd ..
mkdir data/utk
mv utk_images.zip data/utk/
cd data/utk
unzip utk_images.zip
cd ../..
Similarly, for generating one image:
Conda_path/bin/python test_single.py # generating one image at once
--input_img ./data/utk/utk_images/0000.jpg # input image, you could also change this to any input image (0001.jpg, 0002.jpg and etc)
--ref_img ./data/utk/utk_images/0001.jpg # reference image, you could also change this to any reference image (0001.jpg, 0002.jpg and etc)
--checkpoint_dir ./checkpoint/utk # checkpoint path
--result_img ./demo_utk.jpg # result image
For generating multiple images using different reference images:
Conda_path/bin/python generate_multiple.py # generate combinations of reference images, one combination has 5 different reference images, and we randomly use one image as input image.
--data_path ./data/utk/utk_images # images path for providing reference images
--save_path ./data/utk/combination # path to save image combinations
--n_ref # how many reference images in one image combination, we set default is 5
Conda_path/bin/python test_multiple.py
--input_dir ./data/utk/combination # input image path
--checkpoint_dir ./checkpoint/utk # checkpoint path
--result_dir ./result/utk # result path
To make it easier to show the synthesized images, we also provide the merge_result.py to merge the generated images to a larger image.
Conda_path/bin/python merge_results.py
--data_path ./result/utk # the results path
--save_path ./result/utk/merge # the path to save merged images.
We merge 96 synthesized images to a large image, and the middle enlarged image is the input image.
If you want to test your own images or train your model, you can use extract_landmark.py to get facial landmark of face images. We use dlib to obtain the 68 facial landmarks. You should follow the instructions to install dlib.
python extract_landmark.py # the python path should be the path you installed dlib
--data_path # the real RGB path
--save_path # path to save landmark images
--model_path # the predictor path mentioned in dlib, default is shape_predictor_68_face_landmarks.datAfter extracting the landmarks, we also provide a post-processing code facial_normalized.py to get center cropped image.
For the skeleton or pose extraction, we use Openpose to obtain skeleton image. Or you can use Densepose, which maybe helps a better performance for our task.
Reshaping datasets:
- KDEF
- RaFD
- FEI
- CelebA
- UTKFace
- Yale
- WSEFEP
- ADFES
- IIIT-CFW
- PHOTO-SKETCH
- CUHK
- Face Sketch database
├── demo
├── train
├── 000001.jpg
├── 000002.jpg
└── ...
├── test
├── a.jpg (The test image that you want)
├── b.png
└── ...
├── demo
├── train
├── 000001.jpg
├── 000002.jpg
└── ...
├── test
├── a.jpg (The test image that you want)
├── b.png
└── ...
├── demo
├── train
├── 000001.jpg
├── 000002.jpg
└── ...
├── test
├── a.jpg (The test image that you want)
├── b.png
└── ...
├── attribute.txt (For domain attribute information for training)
├── attribute_test.txt (For domain attribute information for training)
DRAGAN: Perturbed loss.LSGAN: Least Square GAN.
Codes will be released soon!
The 5 × 5 outputs by our ReshapeGAN using random 5 images as both inputs and references on CelebA dataset.
Random 96 synthesized images by our ReshapeGAN from one input sample on CelebA dataset. The middle enlarged image shows the input.
Random 96 generated results by our ReshapeGAN from one input sample on UTKFace dataset. The middle enlarged image shows the input.
Random 96 synthesized images by our ReshapeGAN from one input sample on Cat head dataset. The middle enlarged image shows the input.
Random 96 synthesized images by our ReshapeGAN from one input sample on Panama dataset. The middle enlarged image shows the input.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent