gordonren / pose2pose Goto Github PK

This is a pix2pix demo that learns from pose and translates this into a human. A webcam-enabled application is also provided that translates your pose to the trained pose. Everybody dance now !

License: MIT License

Python 100.00%

pose2pose pix2pix gan generative-adversarial-networks face2face openpose pose-estimation alphapose densepose vid2vid

pose2pose's Introduction

pose2pose-demo

This is a pix2pix demo that learns from pose and translates this into a human. A webcam-enabled application is also provided that translates your pose to the trained pose.

Getting Started

1. Prepare Environment

# Clone this repo
git clone [email protected]:GordonRen/pose2pose.git

# Create the conda environment from file
conda env create -f environment.yml

2. Configure PyOpenPose

https://github.com/FORTH-ModelBasedTracker/PyOpenPose

3. Generate Training Data

python generate_train_data.py --file Panama.mp4

Input:

file is the name of the video file from which you want to create the data set.

Output:

Two folders original and landmarks will be created.

If you want to download my dataset, here is also the video file that I used and the generated training dataset (1427 images already split into training and validation).

4. Train Model

# Clone the repo from Christopher Hesse's pix2pix TensorFlow implementation
git clone https://github.com/affinelayer/pix2pix-tensorflow.git

# Move the original and landmarks folder into the pix2pix-tensorflow folder
mv pose2pose/landmarks pose2pose/original pix2pix-tensorflow/photos_pose

# Go into the pix2pix-tensorflow folder
cd pix2pix-tensorflow/

# Reset to april version
git reset --hard d6f8e4ce00a1fd7a96a72ed17366bfcb207882c7

# Resize original images
python tools/process.py \
  --input_dir photos_pose/original \
  --operation resize \
  --output_dir photos_pose/original_resized
  
# Resize landmark images
python tools/process.py \
  --input_dir photos_pose/landmarks \
  --operation resize \
  --output_dir photos_pose/landmarks_resized
  
# Combine both resized original and landmark images
python tools/process.py \
  --input_dir photos_pose/landmarks_resized \
  --b_dir photos_pose/original_resized \
  --operation combine \
  --output_dir photos_pose/combined
  
# Split into train/val set
python tools/split.py \
  --dir photos_pose/combined
  
# Train the model on the data
python pix2pix.py \
  --mode train \
  --output_dir pose2pose-model \
  --max_epochs 1000 \
  --input_dir photos_pose/combined/train \
  --which_direction AtoB

For more information around training, have a look at Christopher Hesse's pix2pix-tensorflow implementation.

5. Export Model

First, we need to reduce the trained model so that we can use an image tensor as input:
```
python reduce_model.py --model-input pose2pose-model --model-output pose2pose-reduced-model
```
Input:
- model-input is the model folder to be imported.
- model-output is the model (reduced) folder to be exported.
Output:
- It returns a reduced model with less weights file size than the original model.
Second, we freeze the reduced model to a single file.
```
python freeze_model.py --model-folder pose2pose-reduced-model
```
Input:
- model-folder is the model folder of the reduced model.
Output:
- It returns a frozen model file frozen_model.pb in the model folder.

I have uploaded a pre-trained frozen model here. This model is trained on 1427 images with epoch 1000.

6. Run Demo

python pose2pose.py --source 0 --show 2 --tf-model pose2pose-reduced-model/frozen_model.pb

Input:

source is the device index of the camera (default=0).
show is an option to display: 0 shows the normal input; 1 shows the pose; 2 shows the normal input and pose (default=2).
tf-model is the frozen model file.

Example:

Requirements

Acknowledgments

Kudos to Christopher Hesse for his amazing pix2pix TensorFlow implementation and Gene Kogan for his inspirational workshop.
Inspired by Dat Tran.

License

See LICENSE for details.

pose2pose's People

Stargazers

Watchers

pose2pose's Issues

You forget to import argparse in generate_train_data.py

Problem with using generate_train_data.py

Hi, I want to use this program for study. I have used the anaconda, and build Openpose and Pyopenpose, but when I run the python script I get error as follow:

Traceback (most recent call last):
File "generate_train_data.py", line 77, in
run()
File "generate_train_data.py", line 17, in run
False, OP.OpenPose.ScaleMode.ZeroToOne, with_face, with_hands)
Boost.Python.ArgumentError: Python argument types in
OpenPose.init(OpenPose, tuple, tuple, tuple, str, str, int, bool, ScaleMode, bool, bool)
did not match C++ signature:
init(object*, cv::Size, cv::Size_, cv::Size_, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int, bool, OpenPoseWrapper::ScaleMode, bool, bool, int)
init(object*, cv::Size, cv::Size, cv::Size, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int, bool, OpenPoseWrapper::ScaleMode, bool, bool)
init(object*, cv::Size, cv::Size, cv::Size, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int, bool, OpenPoseWrapper::ScaleMode, bool)
init(object*, cv::Size, cv::Size, cv::Size, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int, bool, OpenPoseWrapper::ScaleMode)
init(object*, cv::Size, cv::Size, cv::Size, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int, bool)
init(object*, cv::Size, cv::Size, cv::Size, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, int)
init(object*, cv::Size, cv::Size, cv::Size)
init(_object*)
init(_object*)

How to save keypoints as a jason file?

I would like to save keypoints output as a jason file. How can I do that?

Trouble Running generate_train_data.py

Does this error hold meaning for anyone? Any help would be appreciated - let me know if there's any system info you'd like me to provide. Thanks for the great repo!

/root/PyOpenPose/PyOpenPoseLib/OpenPoseWrapper.cpp:OpenPoseWrapper():83
Starting initialization on thread. In src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():173
E0914 18:49:58.493957  2807 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.
E0914 18:49:58.494484  2807 common.cpp:121] Cannot create Curand generator. Curand won't be available.
F0914 18:49:58.494611  2807 common.cpp:152] Check failed: error == cudaSuccess (35 vs. 0)  CUDA driver version is insufficient for CUDA runtime version
*** Check failure stack trace: ***
    @     0x7f2e8dfa35cd  google::LogMessage::Fail()
    @     0x7f2e8dfa5433  google::LogMessage::SendToLog()
    @     0x7f2e8dfa315b  google::LogMessage::Flush()
    @     0x7f2e8dfa5e1e  google::LogMessageFatal::~LogMessageFatal()
    @     0x7f2e8e77ceb2  caffe::Caffe::SetDevice()
    @     0x7f2e905a947e  op::NetCaffe::initializationOnThread()
    @     0x7f2e905b69ab  op::addCaffeNetOnThread()
    @     0x7f2e905b746c  op::PoseExtractorCaffe::netInitializationOnThread()
    @     0x7f2e905af1f0  op::PoseExtractor::initializationOnThread()
    @     0x7f2e913886e0  OpenPoseWrapper::OpenPoseWrapper()
    @     0x7f2e9139526b  boost::python::objects::make_holder<>::apply<>::execute()
    @     0x7f2e913968b1  boost::python::detail::caller_arity<>::impl<>::operator()()
    @     0x7f2e911425cd  boost::python::objects::function::call()
    @     0x7f2e911427c8  (unknown)
    @     0x7f2e9114a613  boost::python::handle_exception_impl()
    @     0x7f2e9113f999  (unknown)
    @           0x4b0cb3  PyObject_Call
    @           0x4f492e  (unknown)
    @           0x4b0cb3  PyObject_Call
    @           0x4f46a7  (unknown)
    @           0x4b670c  (unknown)
    @           0x4b0cb3  PyObject_Call
    @           0x4c9faf  PyEval_EvalFrameEx
    @           0x4c9d8f  PyEval_EvalFrameEx
    @           0x4c2765  PyEval_EvalCodeEx
    @           0x4c2509  PyEval_EvalCode
    @           0x4f1def  (unknown)
    @           0x4ec652  PyRun_FileExFlags
    @           0x4eae31  PyRun_SimpleFileExFlags
    @           0x49e14a  Py_Main
    @     0x7f2e91ef1830  __libc_start_main
    @           0x49d9d9  _start
Aborted (core dumped)

What is .yml ?

conda env create -f environment.yml

What does environment.yml mean?