Git Product home page Git Product logo

diffface's Introduction

DiffFace: Diffusion-based Face Swapping with Facial Guidance

Official PyTorch implementation of Diffusion-based Face Swapping with Facial Guidance

Environment setup

git clone https://github.com/hxngiee/DiffFace.git
cd DiffFace
conda create -n DiffFace python=3.9.7
conda activate DiffFace

conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 -c pytorch 

pip install -r requirements.txt

Download Pretrained Weights

The weights required for the execution of our DiffFace can be downloaded from link.

mkdir checkpoints
mv Arcface.tar checkpoints/ 
mv FaceParser.pth checkpoints/ 
mv GazeEstimator.pt checkpoints/
mv Model.pt checkpoints/

Directories structure

The dataset and checkpoints should be placed in the following structures below

DiffFace
├── checkpoints
    ├── Arcface.tar
    ├── FaceParser.pth
    ├── GazeEstimator.pt
    └── Model.pt
├── data
    └── src
        ├── 001.png
        └── ...
    └── targ
        ├── 001.png
        └── ...
├── models
├── optimization
├── utils
└── main.py

Quick Inference Using Pretrained Model

Place source and target images in data/src, and data/targ. Then run the following.

python main.py --output_path output/example

License

This licence allows for academic and non-commercial purpose only. The entire project is under the CC-BY-NC 4.0 license.

Citation

If you find DiffFace useful for your work please cite:

@inproceedings{kim2022diffface,
  title = {DiffFace: Diffusion-based Face Swapping with Facial Guidance},
  author = {K. Kim, Y. Kim, S. Cho, J. Seo, J. Nam, K. Lee, S. Kim, K. Lee},
  journal = {Arxiv},
  year = {2022}
}

Acknowledgments

This code borrows heavily from Blended-Diffusion and Guided-Diffusion.

diffface's People

Contributors

hxngiee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

diffface's Issues

Question about your model

Is your model brings pretraied DDPM(trained with FFHQ) and finetune with loss? In your code, it seems like you finetune the model per each image using Model.pt, but there is no word about finetune in your paper. Am i right?
Or, is it updating noise like using classifier guidance?
Thank you for your nice work, and it helped me a lot.

urlopen error Tunnel connection failed: 403 Forbidden

File "/workspace/user_code/swapface_pro/DiffFace/main.py", line 6, in
image_editor = ImageEditor(args)
File "/workspace/user_code/swapface_pro/DiffFace/optimization/image_editor.py", line 123, in init
self.fa = face_alignment.FaceAlignment(face_alignment.LandmarksType.TWO_D, flip_input=False)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/site-packages/face_alignment/api.py", line 77, in init
self.face_detector = face_detector_module.FaceDetector(device=device, verbose=verbose, **face_detector_kwargs)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/site-packages/face_alignment/detection/sfd/sfd_detector.py", line 24, in init
model_weights = load_url(models_urls['s3fd'])
File "/data/miniconda3/envs/DiffFace/lib/python3.9/site-packages/torch/hub.py", line 586, in load_state_dict_from_url
download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/site-packages/torch/hub.py", line 452, in download_url_to_file
u = urlopen(req)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/data/miniconda3/envs/DiffFace/lib/python3.9/urllib/request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>
hi, i have this issue,how can i resolve it

DDIM sampling

Does the current released code version support DDIM sampling? I have tried to tag the "--ddim", but got weird results

LOOKING FOR RELEASE

Thank you for your contribution! Please release the code and models of DiffFace.

AttributeError: 'PosixPath' object has no attribute 'with_stem'

Traceback (most recent call last):
File "main.py", line 7, in
image_editor.edit_image_by_prompt()
File "/root/DiffFace/optimization/image_editor.py", line 393, in edit_image_by_prompt
visualization_path = visualization_path.with_stem(f"{visualization_path.stem}i{iteration_number}b{0}")
AttributeError: 'PosixPath' object has no attribute 'with_stem'

[Training Scripts release]

Dear authors,

Thanks for releasing this project, I do admire the work you guys have done!
I kindly ask you to release the training code, which is important for the community to test ideas and thoughts.

(Even a minimal scripts for training is adequate.)

Best regards!

face alignment method?

Hello, author, thank you for sharing. I have a question: Do you have a specific method for aligning the faces of the source image and the target image? Looking forward to your early reply. Thanks!

Error "AttributeError: _2D"

Traceback (most recent call last):
File "/data/docker/yerang/DiffFace/main.py", line 6, in
image_editor = ImageEditor(args)
File "/data/docker/yerang/DiffFace/optimization/image_editor.py", line 123, in init
self.fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=False)
File "/opt/conda/lib/python3.9/enum.py", line 429, in getattr
raise AttributeError(name) from None
AttributeError: _2D

Solution:
Replace image_editor.py file
"_2D" -> "TWO_D"

It works!!!!!! 👍👍🔥

how to swap hair also?

I modify the parsing index with [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,17],including 17 hair, however, the results seems no diff, can you help me with that?

Some question in Facial Guidance Module

Hey, thanks for your excellent work, i have some question about Facial Guidance Module.
Have you retrain the model(identity guidance mdoel、semantic guidance model、gaze guidance model) on noisy images Xt, or you just using the pretained model in original image X0 for guidance.
Looking forward to your reply.Thanks.

model performance

it seems that the model didn't work well when source and target face have different mask orientation.
output_i_0_b_0

How much VRAM is required?

Thanks for releasing the code & models.
I'm getting cuda memory error on 8gb card. What is the expected Vram?
Setting device to 'cpu' causes error.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.