Comments (7)
kunncheng
commented on June 22, 2024
5
关于LNet的训练过程目前可以参考Wav2Lip，我们与其类似采用self-reconstruction的方式在LRS2 dataset上训练。
迁移到不同数据集上训练有一定困难，若是从网络上收集的数据首先需要进行音视频对齐，其次训练lip-sync判别器，最后训练lip-sync network，具体可以参考这里。
from video-retalking.
kunncheng
commented on June 22, 2024
3
@kunncheng 看SadTalker的项目说是在VoxCeleb1 数据集上训练的，感觉中文的唇动效果似乎要比video-retalking效果好一些。不知道是否有计划提供一些其他数据集上进行训练的模型。
SadTalker是驱动单张图像，本项目是编辑视频，多帧与单帧任务之间难度不同，这也是DNet所要解决的问题，希望能将多帧驱动简化为单帧，即将口型归一化。
也尝试过在别的数据集上训练，但难以收敛或性能未取得明显提升。因此暂时没有该计划
from video-retalking.
kunncheng
commented on June 22, 2024
训练所用的数据集为英文，可以泛化到不同语种，但性能有一定程度的下降。
将LNet在合适的大规模中文视频数据集上重新训练或许能提升效果。
from video-retalking.
zhangziliang04
commented on June 22, 2024
训练部分的程序，是否有计划释放出来。目前为止，尚未看到。不知道是否与有计划开放，训练不成的程序？
from video-retalking.
magicleo
commented on June 22, 2024
@kunncheng 看SadTalker的项目说是在VoxCeleb1 数据集上训练的，感觉中文的唇动效果似乎要比video-retalking效果好一些。不知道是否有计划提供一些其他数据集上进行训练的模型。
from video-retalking.
desaltsand
commented on June 22, 2024
OpenTalker
from video-retalking.
marsjin
commented on June 22, 2024
希望有个中文的训练集；目前适配中文，面部相对清晰度下降不少
from video-retalking.
Related Issues (20)
- Lips don't sync on big faces. HOT 1
- ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor' HOT 9
- RuntimeError: PytorchStreamReader failed reading file data/36: file read failed HOT 2
- how to generate video in realtime and restream to RTMP or WebRTC server/app
- Need to be able to lipsync videos having some frames having no Face HOT 4
- Where do I enter the --one_shot parameter?
- Share my Dockerfile
- how to make the mouth region smaller? HOT 2
- 1080p视频嘴部闪动 HOT 1
- 感觉中文支持比较差 有没有中文的模型 怎么训练模型
- Who needs high-quality lip sync - contact me! HOT 5
- Who needs high-quality lip sync - contact me! HOT 6
- Probably a small bug in gpen_face_enhancer.py HOT 1
- fused/fused.so
- Hi I am new to open source
- Could you please share me the training code?
- it would be good if it's working with just text input
- No output after FaceDet : 100%,^C HOT 1
- ImportError: cannot import name 'packaging' from 'pkg_resources' HOT 4
- Issue if the Video start with a pause.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from video-retalking.