This repo is an unofficial demo web app of the face video2video.

The notebooks are heavily based on this code, created by zhanglonghao1992 (thank you!).

• You can run this demo app in the notebook ✨
• 🎉 Add Colab VER

ex 1	ex 2

Control yaw, pitch, roll (range(-30, 30))

• Python 3.6 and Pytorch 1.7 are used.

2021.11.05 :

• Replace Jacobian with the rotation matrix (Assuming J = R) to avoid estimating Jacobian. (not working)
• Correct the rotation matrix.

2021.11.17 :

• Better Generator, better performance (models and checkpoints have been released).

Driving | Beta Version | FOMM | New Version:

Driving | FOMM | Ours:

Free-View:

python run.py --config config/vox-256.yaml --device_ids 0,1,2,3,4,5,6,7

python demo.py --config config/vox-256.yaml --checkpoint path/to/checkpoint --source_image path/to/source --driving_video path/to/driving --relative --adapt_scale --find_best_frame

free-view (e.g. yaw=20, pitch=roll=0):

python demo.py --config config/vox-256.yaml --checkpoint path/to/checkpoint --source_image path/to/source --driving_video path/to/driving --relative --adapt_scale --find_best_frame --free_view --yaw 20 --pitch 0 --roll 0

Note: run crop-video.py --inp driving_video.mp4 first to get the cropping suggestion and crop the raw video.

Note:

1. For now, the Beta Version is not well tuned.
2. For free-view synthesis, it is recommended that Yaw, Pitch and Roll are within ±45°, ±20° and ±20° respectively.
3. Face Restoration algorithms (GPEN) can be used for post-processing to significantly improve the resolution.

Thanks to NV, AliaksandrSiarohin and DeepHeadPose.