MotionBERT: Unified Pretraining for Human Motion Analysis

Project Page | Paper

This is the official PyTorch implementation of the paper "MotionBERT: Unified Pretraining for Human Motion Analysis".

Installation

conda create -n motionbert python=3.7 anaconda
conda activate motionbert
# Please install PyTorch according to your CUDA version.
conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia
pip install -r requirements.txt

Usage

Using MotionBERT for new tasks

'''	    
  x: 2D skeletons 
    type = <class 'torch.Tensor'>
    shape = [batch size * frames * joints(17) * channels(3)]
    
  MotionBERT: pretrained MotionBERT
    type = <class 'lib.model.DSTformer.DSTformer'>
    
  E: encoded motion representation
    type = <class 'torch.Tensor'>
    shape = [batch size * frames * joints(17) * channels(512)]
'''
E = MotionBERT.get_representation(x)

Hints

1. The model could handle different input lengths (no more than 243 frames). No need to explicitly specify the input length elsewhere.
2. The model uses 17 body keypoints (H36M format). If you are using other formats, please convert them before feeding to MotionBERT.
3. Please refer to model_action.py and model_mesh.py for examples of (easily) adapting MotionBERT to different downstream tasks.

Model Zoo

TODO

• Scripts and docs for pretraining

• Demo for custom videos

BibTeX

If you find our work useful for your research, please consider citing the paper:

@article{motionbert2022,
  title   =   {MotionBERT: Unified Pretraining for Human Motion Analysis}, 
  author  =   {Zhu, Wentao and Ma, Xiaoxuan and Liu, Zhaoyang and Liu, Libin and Wu, Wayne and Wang, Yizhou},
  year    =   {2022},
  journal =   {arXiv preprint arXiv:2210.06551},
}