Git Product home page Git Product logo

xinggan's Introduction

License CC BY-NC-SA 4.0 Python 3.6 Packagist Last Commit Maintenance Contributing Ask Me Anything !

Contents

XingGAN or CrossingGAN

| Project | Paper |
XingGAN for Person Image Generation
Hao Tang12, Song Bai2, Li Zhang2, Philip H.S. Torr2, Nicu Sebe13.
1University of Trento, Italy, 2University of Oxford, UK, 3Huawei Research Ireland, Ireland.
In ECCV 2020.
The repository offers the official implementation of our paper in PyTorch.

In the meantime, check out our related ACM MM 2019 paper Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation, BMVC 2020 oral paper Bipartite Graph Reasoning GANs for Person Image Generation, and ICCV 2021 paper Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer.

Framework

Comparison Results


License

Creative Commons License
Copyright (C) 2020 University of Trento, Italy.

All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)

The code is released for academic research use only. For commercial use, please contact [email protected].

Installation

Clone this repo.

git clone https://github.com/Ha0Tang/XingGAN
cd XingGAN/

This code requires PyTorch 1.0.0 and python 3.6.9+. Please install the following dependencies:

  • pytorch 1.0.0
  • torchvision
  • numpy
  • scipy
  • scikit-image
  • pillow
  • pandas
  • tqdm
  • dominate

To reproduce the results reported in the paper, you need to run experiments on NVIDIA DGX1 with 4 32GB V100 GPUs for DeepFashion, and 1 32GB V100 GPU for Market-1501.

Dataset Preparation

Please follow SelectionGAN to directly download both Market-1501 and DeepFashion datasets.

This repository uses the same dataset format as SelectionGAN and BiGraphGAN. so you can use the same data for all these methods.

Generating Images Using Pretrained Model

Market-1501 (Run twice, the first time create a new folder!!!)

sh scripts/download_xinggan_model.sh market
sh scripts/download_xinggan_model.sh market

Then,

  1. Change several parameters in test_market.sh.
  2. Run sh test_market.sh for testing.

DeepFashion

sh scripts/download_xinggan_model.sh deepfashion

Then,

  1. Change several parameters in test_deepfashion.sh.
  2. Run sh test_deepfashion.sh for testing.

Train and Test New Models

Market-1501

  1. Change several parameters in train_market.sh.
  2. Run sh train_market.sh for training.
  3. Change several parameters in test_market.sh.
  4. Run sh test_market.sh for testing.

DeepFashion

  1. Change several parameters in train_deepfashion.sh.
  2. Run sh train_deepfashion.sh for training.
  3. Change several parameters in test_deepfashion.sh.
  4. Run sh test_deepfashion.sh for testing.

Evaluation

We adopt SSIM, mask-SSIM, IS, mask-IS, and PCKh for evaluation of Market-1501. SSIM, IS, PCKh for DeepFashion.

  1. SSIM, mask-SSIM, IS, mask-IS: install python3.5, tensorflow 1.4.1, and scikit-image==0.14.2. Then run, python tool/getMetrics_market.py or python tool/getMetrics_fashion.py.

  2. PCKh: install python2, and pip install tensorflow==1.4.0, then set export KERAS_BACKEND=tensorflow. After that, run python tool/crop_market.py or python tool/crop_fashion.py. Next, download pose estimator and put it under the root folder, and run python compute_coordinates.py. Lastly, run python tool/calPCKH_market.py or python tool/calPCKH_fashion.py.

Please refer to Pose-Transfer for more details.

Acknowledgments

This source code is inspired by both Pose-Transfer and SelectionGAN.

Related Projects

BiGraphGAN | GestureGAN | C2GAN | SelectionGAN | Guided-I2I-Translation-Papers

Citation

If you use this code for your research, please consider giving a star ⭐ and citing our paper 🦖:

XingGAN

@inproceedings{tang2020xinggan,
  title={XingGAN for Person Image Generation},
  author={Tang, Hao and Bai, Song and Zhang, Li and Torr, Philip HS and Sebe, Nicu},
  booktitle={ECCV},
  year={2020}
}

If you use the original BiGraphGAN, GestureGAN, C2GAN, and SelectionGAN model, please consider giving stars ⭐ and citing the following papers 🦖:

BiGraphGAN

@article{tang2022bipartite,
  title={Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesis},
  author={Tang, Hao and Shao, Ling and Torr, Philip HS and Sebe, Nicu},
  journal={International Journal of Computer Vision (IJCV)},
  year={2022}
}

@inproceedings{tang2020bipartite,
  title={Bipartite Graph Reasoning GANs for Person Image Generation},
  author={Tang, Hao and Bai, Song and Torr, Philip HS and Sebe, Nicu},
  booktitle={BMVC},
  year={2020}
}

GestureGAN

@article{tang2019unified,
  title={Unified Generative Adversarial Networks for Controllable Image-to-Image Translation},
  author={Tang, Hao and Liu, Hong and Sebe, Nicu},
  journal={IEEE Transactions on Image Processing (TIP)},
  year={2020}
}

@inproceedings{tang2018gesturegan,
  title={GestureGAN for Hand Gesture-to-Gesture Translation in the Wild},
  author={Tang, Hao and Wang, Wei and Xu, Dan and Yan, Yan and Sebe, Nicu},
  booktitle={ACM MM},
  year={2018}
}

C2GAN

@article{tang2021total,
  title={Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes},
  author={Tang, Hao and Sebe, Nicu},
  journal={IEEE Transactions on Multimedia (TMM)},
  year={2021}
}

@inproceedings{tang2019cycleincycle,
  title={Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation},
  author={Tang, Hao and Xu, Dan and Liu, Gaowen and Wang, Wei and Sebe, Nicu and Yan, Yan},
  booktitle={ACM MM},
  year={2019}
}

SelectionGAN

@article{tang2022multi,
  title={Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation},
  author={Tang, Hao and Torr, Philip HS and Sebe, Nicu},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year={2022}
}


@inproceedings{tang2019multi,
  title={Multi-channel attention selection gan with cascaded semantic guidance for cross-view image translation},
  author={Tang, Hao and Xu, Dan and Sebe, Nicu and Wang, Yanzhi and Corso, Jason J and Yan, Yan},
  booktitle={CVPR},
  year={2019}
}

Contributions

If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Hao Tang ([email protected]).

Collaborations

I'm always interested in meeting new people and hearing about potential collaborations. If you'd like to work together or get in contact with me, please email [email protected]. Some of our projects are listed here.

Progress is impossible without change, and those who cannot change their minds cannot change anything.

xinggan's People

Contributors

ha0tang avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

samjcheng a489369729 xrosliang peternara hephaex lotayou ml-lab jaedukseo pinglmlcv tobran sunshinewhy jimmyrslab hwb0314 gaokun556 lisiyaoatbnu gaolf gwliu213 tchigher c00renut elnaaz rakaiv5 cv-ip ritesh-nanda wesias8 orlgln templeblock xinyouduogao lockejiang luking yhlleo dogoulis stream122 masom9768 suqiguang jingli-wtbox

xinggan's Issues

Some question about the SSIM metric

Thanks for your novel work. When I calculated the SSIM metric, it's a little lower than the value that in the paper.
Here are the results that I obtained.
Market1501 dataset
image

Some visual results that obtained with the pretrained model.
/0265_c6s1_056351_01.jpg___0265_c3s1_062642_05.jpg_vis.jpg
image
1322_c3s3_035678_02.jpg___1322_c4s5_061435_01.jpg_vis.jpg
image

I don't know where I got it wrong.
Thanks for your time.

License

Under what license are you releasing this code?

How to visualize the pose map?

Thanks for your novel work. It helps me a lot! And I wonder how to visualize the pose map as the figure shown in your paper? I can't find the visualization process in this repo.

截屏2021-01-06 下午5 09 49

Accuracy without Co-attention Fusion Module

Thanks for this great work, you have mentioned the accuracy with both SA and AS blocks but under the absence of the co-attention fusion module in the paper and I wonder how did you get the result in this case? Did you have a direct FC layer at the end of the attention modules? How can we replicate that result?

Image size does not match to the pose heat map size

Hi,
Thanks for your great work.
When I load the deepfashion dataset, I found the size of image is 256X256 but the size of the heat map is 256X176.
How can I use this data? Do I need to resize the image?

Custom datasets

Thanks for your excellent work. I want to conduct experiments on my own dataset. May I ask how to prepare my own dataset.

Training Problems

@Ha0Tang Thanks for your novel work. I have trained the marker dataset follow your guidance. But I have a question that With the increase of training iteration, the loss of each part also increases except D_PP and D_PB。what are the two parts mean? I also wanted to ask how many epochs were used in the pre-training models you provided
image
image

reproducing results using pretrained Deep Fashion Model -- quality seems not as good as expected

Dear paper authors

I am working on a NeurIPS paper for PoseMorphing, so I wanted a good comparison with your state-of-the-art method.

I tried to run your pretrained deepfashion model, and it worked. However, the results seem worse than I expected, can it be that the latest pytorch 1.7 has broken some detail in your version?

I write the command I used to run your model, and 2 example results. Maybe you can tell me if they look as expected, or something has gone wrong?

python test.py --dataroot deepfashion/ --name deepfashion_XingGAN --model XingGAN --phase test --dataset_mode keypoint --norm instance --batchSize 1 --resize_or_crop no --gpu_ids 0 --BP_input_nc 18 --no_flip --which_model_netG Xing --checkpoints_dir ./checkpoints --pairLst /deepfahion/fasion-resize-pairs-test.csv --which_epoch latest --results_dir ./results --display_id 0

fashionMENTees_Tanksid0000730104_7additional jpg___fashionMENTees_Tanksid0000730104_1front jpg_vis
fashionWOMENBlouses_Shirtsid0000337203_3back jpg___fashionWOMENBlouses_Shirtsid0000337203_2side jpg_vis

fashionMENTees_Tanksid0000730104_7additional jpg___fashionMENTees_Tanksid0000730104_1front jpg_vis

fashionWOMENBlouses_Shirtsid0000337203_3back jpg___fashionWOMENBlouses_Shirtsid0000337203_2side jpg_vis

thanks a lot for your help to make research reproducible

Start point

Hi Hao

I'm trying to download the images but keep getting the following error:

Forbidden

You don't have permission to access /~hao.tang/uploads/models/XingGAN/ on this server.
Apache/2.4. .... Server at disi.unitn.it Port 80

May you please help where might be the problem?

Face swapping(future work)

Hey @Ha0Tang
Thanks for sharing such awesome work!
I was wondering if we utilize your algorithm and facial landmarks to do face swapping and generate talking head models?

estimator not work

Hi, why I always get [-1, -1] * 18 when I try to run compute_coordinates.py? Such as:
0000_c6s3_047142_03.jpg: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
0609_c5s2_022005_05.jpg: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
0304_c5s1_068698_01.jpg: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
-1_c6s4_006527_05.jpg: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]: [-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
...
For any images I test, the cooridiantes always are [-1, -1] * 18, how can I solve this error?
Thanks a lot!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.