Git Product home page Git Product logo

lip_mask's Introduction

LIP_MASK:具有地标和外观先验的保持身份的说话人脸生成(CVPR 2023)

CVPR2023论文“Identity-Preserving Talking Face Generation with Landmark and Appearance Priors” 的PyTorch官方方案的升级。

[论文] [演示视频]

如何实现效果的提升?

  • 通过dlib优化关键点检测方式
  • 对齐音频视频帧
  • 二次生成说话脸速度提升
  • 通过锐化滤波加强清晰度

环境要求

  • Python 3.9
  • torch 2.0.0
  • torchvision 0.15.1
  • ffmpeg

我们在1个24G的RTX3090上使用CUDA 118进行实验。更多细节,请参考 requirements.txt。我们建议首先安装pytorch,然后运行以下命令:

pip install -r requirements.txt

测试

FoxCloud下载预训练模型,密码'foxyear',并将其放置在 checkpoints 文件夹中。然后运行以下命令:

python inference.py

运行

FoxCloud下载预训练模型,密码'foxyear',并将其放置在 checkpoints 文件夹中。然后运行以下命令:

python inference.py  --input ./video/videoxx.mp4 --audio ./audio/testxx.wav

致谢

该项目在公开可用的代码 IP_LAP , DFRF , pix2pixHD, vico_challengeWav2Lip 基础上构建而成。感谢这些作品和代码的作者将他们优秀的工作公开发布。

联系

深度开发合作交流,联系加微信:

交流群及资料教程:

引用和点赞

如果你在研究中使用了这个库,请引用以下论文并为该项目点赞。谢谢!

@InProceedings{Zhong_2023_CVPR,
    author    = {Zhong, Weizhi and Fang, Chaowei and Cai, Yinqi and Wei, Pengxu and Zhao, Gangming and Lin, Liang and Li, Guanbin},
    title     = {Identity-Preserving Talking Face Generation With Landmark and Appearance Priors},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {9729-9738}
}

lip_mask's People

Contributors

foxyear-kyumin avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.