$ git clone https://github.com/Jhuangsp/pseudo3d.git
$ cd pseudo3d
$ git clone https://github.com/Jhuangsp/h2pose.git
$ pip install numpy
$ pip install opencv-python
$ pip install pandas
$ pip install Pillow
$ pip install PyYAML
$ pip install scikit-learn
# [For windows] go to https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopengl
# and download PyOpenGL-3.1.5-cp38-cp38-win_amd64.whl
$ pip install PyOpenGL-3.1.5-cp38-cp38-win_amd64.whl
# [For ubuntu]
$ apt-get install freeglut3-dev
$ pip install PyOpenGL
For the synthetic_pseudo3D.py
demo, first point out the 4 corner of the badminton court in order of left-top
, right-top
, right-bottom
, left-bottom
.
If you want to adjust your corner position, please complete the 4 corner first, then adjust the corner by clicking on the new position.
The program will find the nearest corner and modify it with new position.
# Real-scene monocular visualization
$ python pseudo3D.py --track real_track/CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/set_1_00_01.csv --height 720 --width 1280
# Synthetic monocular visualization
$ python synthetic_pseudo3D.py --json synthetic_track/mono/track.json
Use keyboard A/D
to control viewing postiton, W/S
to change guessed focal length, space
(Synthetic Only) to use or not to use the ground truth pose.
Real-scene monocular visualization (left image) and Synthetic monocular visualization (right image)
Our 2D tracking dataset is generated by TrackNetV2 & S2Label, which can download from Google drive.
The dataset contains the 2D trajectory predicetion of each rally e.g. tracknet/1_00_01.csv
, shot-by-shot annotation of each set e.g. set1.csv
and some hyper-annotation such as rally segmentation RallySeg.csv
and homography transform matrix homography_matrix.csv
.
There are some notable things that you should know:
Old format | New format | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Use the Rearrange_Label.py
to transform old format to new format.
# Usage: python Rearrange_Label.py <inputPath> <outputPath> <segFile>
$ cd real_track
$ mkdir new_format
$ python Rearrange_Label.py CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/old_format/ new_format/ CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/RallySeg.csv
We change the original rally-based indexing to set-based indexing, and add the hit event attributes which mark the frame as Hit=True
, (StartX
, StartY
) and (EndX
, EndY
) as the anchor points of start/end of shot.
New format | New format associate with S2 annotation | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
- Note that #10454 frame is the serve frame, which doesn't label the start anchor point of the serve shot.
Use the rallyMatcher.py
to associate annotation.
# Usage: python rallyMatcher.py --rally <RALLY_FOLDER> --set <SET_FILE> --seg <SEGMENT_FILE> --out <OUTPUT_FOLDER>
$ cd real_track
$ mkdir associated
$ python rallyMatcher.py --rallys CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/new_format --sets CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/label --seg CHEN_Long_CHOU_Tien_Chen_Denmark_Open_2019_QuarterFinal/RallySeg.csv --out associated
Generate 1 synthetic image with json pose config
$ mkdir mydata
$ cp synthetic_track/mono/track.json mydata/track.json
$ python generator.py --json mydata/track.json --out mydata/ --num 1
There are several parameters to define a camera pose in this project:
- Camera pose parameters
distance
: Thedistance
parameter sets the distance between camera to the center. Intrack.json
, we set the camera way from the center by 18 meters, and looking at the center (before any shifting).rotation
: Therotation
parameter contains the 3 factorsroll
,yaw
,pitch
, which set the direction from center to camera. Our program will get the pose of camera sampled from the given range of parameter. By default, the direction from center to camera is along with the Y-axis direction of the world coord. system. (roll=0, yaw=0, pitch=0
) (We suggest that setting theroll: [0, 0], yaw: [-45, 45], pitch: [10, 50]
for more realistic scene)shift
: After setting thedistance
androtation
, we will shift the camera along the XY-direction of world coord. system withshift
parameter. (The camera is no more looking at center and the distance between center and camera is changed, too)
- Calibration pattern parameters
pattern
:two_circle
orchessboard
can be set.two_circle
pattern is for Chen et al's "Camera calibration with two arbitrary coplanar circles" method and thechessboard
pattern is for Zhang et al's "A flexible new technique for camera calibration" method.size
: Set the radius of circles or the cube size of chessboard.space
:two_circle
only, Set the space between circles.
For example, the track.json
shows as follow
{
"pattern": "two_circle",
"rotation": {
"roll": [0, 0],
"yaw": [0, 0],
"pitch": [30, 30]
},
"shift": [0, 0],
"size": 0.06,
"distance": 18,
"space": 1.0
}