![](https://private-user-images.githubusercontent.com/78398294/293370037-ad0a79ba-41c4-449b-96ee-a49c7591a94d.gif?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg0Nzc4MDAsIm5iZiI6MTcxODQ3NzUwMCwicGF0aCI6Ii83ODM5ODI5NC8yOTMzNzAwMzctYWQwYTc5YmEtNDFjNC00NDliLTk2ZWUtYTQ5Yzc1OTFhOTRkLmdpZj9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjE1VDE4NTE0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI5MTdlMzZiZmI3OGIxZTBjNDY4YTM5Y2I5NzYyYmUzMzA2OWFkZmI5YjU1YWRlOGQ1MzRmM2ViYzZmNTJkMTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.a4L3-A5jvjMuu1t95P-kkDPgFRCXJIHN5BHrejKc7UI)
![](https://private-user-images.githubusercontent.com/78398294/293370046-ebbab7a8-4182-4d0d-9a4c-b7615a2eeaa8.gif?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg0Nzc4MDAsIm5iZiI6MTcxODQ3NzUwMCwicGF0aCI6Ii83ODM5ODI5NC8yOTMzNzAwNDYtZWJiYWI3YTgtNDE4Mi00ZDBkLTlhNGMtYjc2MTVhMmVlYWE4LmdpZj9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjE1VDE4NTE0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTlhMTYwOTgwZjRiM2Q3ZDg3MGI0NjM3OWNiOTgwNWIyNzIwZmJlNTMxNjJhOTMwM2NjZDNmMzE3ZGM5MzBmZmMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.w1Age8BuCLOQNmV2zZ-UWPPnbiabK7fI0vL08dK_doU)
DreamWaltz extension of threestudio. To use it, please install threestudio first and then install this extension in threestudio custom
directory.
cd custom
git clone https://github.com/huanngzh/threestudio-dreamwaltz.git
cd threestudio-dreamwaltz
pip install -r requirements.txt
If installing the pytorch3d package fails, please see the detailed instructions at pytorch3d/INSTALL.md.
We use smpl and vposer models for avatar creation and animation learning, please follow the instructions in smplx and human_body_prior to download the model weights, and build a directory with the following structure:
smpl_models
├── smpl
│ ├── SMPL_FEMALE.pkl
│ └── SMPL_MALE.pkl
│ └── SMPL_NEUTRAL.pkl
└── vposer
└── v2.0
├── snapshots
├── V02_05.yaml
└── V02_05.log
Then, update the model paths SMPL_ROOT
and VPOSER_ROOT
in utils/smpl/smpl_prompt.py
.
All in one (SMPL Initializaion + Canonical Avatar Creation):
python launch.py --config custom/threestudio-dreamwaltz/configs/dreamwaltz-static.yaml --train --gpu 0 system.prompt_processor.prompt="Naruto"
Divided into multiple stages:
# SMPL Initializaion
python launch.py --config custom/threestudio-dreamwaltz/configs/experimental/dreamwaltz-1-warmup.yaml --train --gpu 0 system.prompt_processor.prompt="Naruto"
# Canonical Avatar Creation
python launch.py --config custom/threestudio-dreamwaltz/configs/experimental/dreamwaltz-2-nerf.yaml --train --gpu 0 system.prompt_processor.prompt="Naruto" resume=path/to/trial/dir/ckpts/last.ckpt
Not yet implemented!
If you find DreamWaltz helpful, please consider citing:
@article{huang2023dreamwaltz,
title={DreamWaltz: Make a Scene with Complex 3D Animatable Avatars},
author={Yukun Huang and Jianan Wang and Ailing Zeng and He Cao and Xianbiao Qi and Yukai Shi and Zheng-Jun Zha and Lei Zhang},
journal = {arXiv:2305.12529},
year={2023},
}