1. Installation
2. Quick Start Demo
3. How to get real scene data
4. How to generate synthetic data

$ git clone https://github.com/Jhuangsp/pseudo3d.git
$ cd pseudo3d
$ git clone https://github.com/Jhuangsp/h2pose.git

$ pip install numpy
$ pip install opencv-python
$ pip install pandas
$ pip install Pillow
$ pip install PyYAML
$ pip install scikit-learn

# [For windows] go to https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopengl
# and download PyOpenGL-3.1.5-cp38-cp38-win_amd64.whl
$ pip install PyOpenGL-3.1.5-cp38-cp38-win_amd64.whl

# [For ubuntu]
$ apt-get install freeglut3-dev
$ pip install PyOpenGL

For the synthetic_pseudo3D.py demo, first point out the 4 corner of the badminton court in order of left-top, right-top, right-bottom, left-bottom. If you want to adjust your corner position, please complete the 4 corner first, then adjust the corner by clicking on the new position. The program will find the nearest corner and modify it with new position.

# Real-scene monocular visualization
$ python pseudo3D.py --track real_track/CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/set_1_00_01.csv --height 720 --width 1280

# Synthetic monocular visualization
$ python synthetic_pseudo3D.py --json synthetic_track/mono/track.json

Use keyboard A/D to control viewing postiton, W/S to change guessed focal length, space(Synthetic Only) to use or not to use the ground truth pose.

^{Real-scene monocular visualization (left image) and Synthetic monocular visualization (right image)}

Our 2D tracking dataset is generated by TrackNetV2 & S2Label, which can download from Google drive. The dataset contains the 2D trajectory predicetion of each rally e.g. tracknet/1_00_01.csv, shot-by-shot annotation of each set e.g. set1.csv and some hyper-annotation such as rally segmentation RallySeg.csv and homography transform matrix homography_matrix.csv.

There are some notable things that you should know:

Use the Rearrange_Label.py to transform old format to new format.

# Usage: python Rearrange_Label.py <inputPath> <outputPath> <segFile>

$ cd real_track
$ mkdir new_format
$ python Rearrange_Label.py CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/old_format/ new_format/ CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/RallySeg.csv

We change the original rally-based indexing to set-based indexing, and add the hit event attributes which mark the frame as Hit=True, (StartX, StartY) and (EndX, EndY) as the anchor points of start/end of shot.

• Note that #10454 frame is the serve frame, which doesn't label the start anchor point of the serve shot.

Use the rallyMatcher.py to associate annotation.

# Usage: python rallyMatcher.py --rally <RALLY_FOLDER> --set <SET_FILE> --seg <SEGMENT_FILE> --out <OUTPUT_FOLDER>

$ cd real_track
$ mkdir associated
$ python rallyMatcher.py --rallys CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/new_format --sets CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/label --seg CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/RallySeg.csv --out associated

Generate 1 synthetic image with json pose config

$ mkdir mydata
$ cp synthetic_track/mono/track.json mydata/track.json
$ python generator.py --json mydata/track.json --out mydata/ --num 1

There are several parameters to define a camera pose in this project:

• Camera pose parameters
  • distance: The distance parameter sets the distance between camera to the center. In track.json, we set the camera way from the center by 18 meters, and looking at the center (before any shifting).
  • rotation: The rotation parameter contains the 3 factors roll, yaw, pitch, which set the direction from center to camera. Our program will get the pose of camera sampled from the given range of parameter. By default, the direction from center to camera is along with the Y-axis direction of the world coord. system. (roll=0, yaw=0, pitch=0) (We suggest that setting the roll: [0, 0], yaw: [-45, 45], pitch: [10, 50] for more realistic scene)
  • shift: After setting the distance and rotation, we will shift the camera along the XY-direction of world coord. system with shift parameter. (The camera is no more looking at center and the distance between center and camera is changed, too)
• Calibration pattern parameters
  • pattern: two_circle or chessboard can be set. two_circle pattern is for Chen et al's "Camera calibration with two arbitrary coplanar circles" method and the chessboard pattern is for Zhang et al's "A flexible new technique for camera calibration" method.
  • size: Set the radius of circles or the cube size of chessboard.
  • space: two_circle only, Set the space between circles.

For example, the track.json shows as follow

{
  "pattern": "two_circle",
  "rotation": {
    "roll": [0, 0],
    "yaw": [0, 0],
    "pitch": [30, 30]
  },
  "shift": [0, 0],
  "size": 0.06,
  "distance": 18,
  "space": 1.0
}