This is an implementation of Realtime Multi-Person Pose Estimation with Chainer. The original project is here.

日本語版 README

 

This project is licensed under the terms of the license.

1. Converting caffe model
2. Testing
3. Training

• Python 3.0+
• Chainer 2.0+
• NumPy
• Matplotlib
• OpenCV

The authors of the original implementation provide trained caffe model which you can use to extract model weights. Execute the following commands to download the trained model and convert it to npz file:

cd models
wget http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel
wget http://posefs1.perception.cs.cmu.edu/OpenPose/models/face/pose_iter_116000.caffemodel
wget http://posefs1.perception.cs.cmu.edu/OpenPose/models/hand/pose_iter_102000.caffemodel
python convert_model.py posenet pose_iter_440000.caffemodel coco_posenet.npz
python convert_model.py facenet pose_iter_116000.caffemodel facenet.npz
python convert_model.py handnet pose_iter_102000.caffemodel handnet.npz
cd ..

Execute the following command with the weight parameter file and the image file as arguments for estimating pose. The resulting image will be saved as result.png.

python pose_detector.py posenet models/coco_posenet.npz --img data/person.png 

If you have a gpu device, use the --gpu option.

python pose_detector.py posenet models/coco_posenet.npz --img data/person.png --gpu 0

 

Similarly, execute the following command for face estimation. The resulting image will be saved as result.png.

python face_detector.py facenet models/facenet.npz --img data/face.png 

 

Similarly, execute the following command for hand estimation. The resulting image will be saved as result.png.

python hand_detector.py handnet models/handnet.npz --img data/hand.png 

 

Similarly, you can detect all poses, faces and hands by executing the following command. The resulting image will be saved as result.png.

python demo.py --img data/dinner.png

 

If you have a web camera, you can execute the following cammand to run real-time demostration mode with the camera activated. Quit with the q key.

Real-time pose estimation:

python camera_pose_demo.py

Real-time face estimation:

python camera_face_demo.py

This is a full scratch training procedure using COCO 2017 dataset.

bash getData.sh

If you already downloaded the dataset by yourself, please skip this procedure and change coco_dir in entity.py to the dataset path that was already downloaded.

git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI/
make
python setup.py install
cd ../../

Execute the following command to check randomly generated training images by generator. Please confirm that you can see the correct PAFs, Heatmaps, and masks on the clipped image.

python coco_data_loader.py

For each 1000 iterations, the recent weight parameters are saved as a weight file model_iter_1000.

python train_coco_pose_estimation.py --gpu 0

Execute the following command with your own trained weight parameter file and the image file as arguments for inference. The resulting image will be saved as result.png.

python pose_detector.py posenet model_iter_1000 --img data/person.png 

• CVPR'16, Convolutional Pose Machines.
• CVPR'17, Realtime Multi-Person Pose Estimation.

Please cite the original paper in your publications if it helps your research:

@InProceedings{cao2017realtime,
  title = {Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields},
  author = {Zhe Cao and Tomas Simon and Shih-En Wei and Yaser Sheikh},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2017}
  }