By: David Novotny, Nikhila Ravi, Benjamin Graham, Natalia Neverova, Andrea Vedaldi
This is the official implementation of C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion in PyTorch.
This is a Python 3.6 package. Required packages can be installed with e.g. pip and conda:
> conda create -n c3dpo python=3.6
> pip install -r requirements.txt
The complete list of dependencies:
- pytorch (version==1.1.0)
- numpy
- tqdm
- matplotlib
- visdom
- pyyaml
- tabulate
demo.py downloads and runs a pre-trained C3DPO model on a sample skeleton from the Human36m dataset and generates a 3D figure with a video of the predicted 3D skeleton:
> python ./demo.py
Note that all the outputs are dumped to a local Visdom server. You can start a Visdom server with:
> python -m visdom.server
Images are also stored to the ./data directory. The video will get exported only if there's a functioning ffmpeg callable from the command line.
Whenever needed, all datasets / pre-trained models are automatically downloaded to various folders under the ./data directory. Hence, there's no need to bother with a complicated data setup :). In case you would like to cache all the datasets for your own use, simply run the evaluate.py which downloads all the needed data during its run.
Pre-trained networks can be evaluated by calling evaluate.py:
> python evaluate.py
Note that we provide pre-trained models that will get auto-downloaded during the run of the script to the ./data/exps/ directory.
Furthermore, the datasets will also be automatically downloaded in case they are not stored in ./data/datasets/.
Launch experiment.py with the argument cfg_file set to the yaml file corresponding the relevant dataset., e.g.:
> python ./experiment.py --cfg_file ./cfgs/h36m.yaml
will train a C3DPO model for the Human3.6m dataset.
Note that the code supports visualisation in Visdom. In order to enable Visdom visualisations, first start a visdom server with:
> python -m visdom.server
The experiment will output learning curves as well as visualisations of the intermediate outputs to the visdom server.
Furthermore, the results of the evaluation will be periodically updated after every training epoch in ./data/exps/c3dpo/<dataset_name>/eval_results.json. The metrics reported in the paper correspond to 'EVAL_MPJPE_best' and 'EVAL_stress'.
For the list of all possible yaml config files, please see the ./cfgs/ directory. Each config .yaml file corresponds to a training on a different dataset (matching the name of the .yaml file). Expected quantitative results are the same as for the evaluate.py script.
If you find our work useful, please cite it using the following bibtex reference.
@inproceedings{novotny2019c3dpo,
title={C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion},
author={Novotny, David and Ravi, Nikhila and Graham, Benjamin and Neverova, Natalia and Vedaldi, Andrea},
booktitle={Proceedings of the IEEE International Conference on Computer Vision},
year={2019}
}
C3DPO is distributed under the MIT license, as found in the LICENSE file.
Below are the results of the supplied pre-trained models for all datasets:
dataset MPJPE Stress
-------------- ----------- ----------
h36m 95.6338 41.5864
h36m_hourglass 145.021 84.693
pascal3d_hrnet 56.8909 40.1775
pascal3d 36.6413 31.0768
up3d_79kp 0.0672771 0.0406902
Note that the models have better performance than published mainly due to letting the models to train for longer.
Note that the performance reported above was obtained with PyTorch v1.1. If you notice differences in performance make sure to use PyTorch v1.1.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent