Please refer to our paper for detailed explanation. Arxiv Link is here.
In the following, ROOT
refers to the folder containing this README file.
I contained all important public literature in this repo to boost research. If there is a conflict or an issue, please raise it in issue part.
- All the codes are tested in Python, CUDA 8.0 and CUDNN 5.1.
- Install pytorch 0.4.0 and other dependencies.
- Download LineMod dataset here. Only folders called models and test are needed. Put them in
DATAROOT/models
andDATAROOT/test
whereDATAROOT
can be any folder you'd like to place LineMod dataset.
You can skip this step since we have provided designated keypoints files in '$ROOT/1_keypoint_designator/assets/sifts/'.
- The related code is in
$ROOT/1_keypoint_designator/
.$ cd ROOT/1_keypoint_designator/
- Place the input model file (e.g.
DATAROOT/models/obj_01.ply
) in$./assets/models/
- Build the code and run it. Just type:
The output file is in
$ sh build_and_run.sh
$./assets/sifts/
. It's a ply file storing the 3D coordinates of designated keypoints.
- The related code is in
$ROOT/2_keypoint_annotator/
.$ cd ROOT/2_keypoint_annotator/
- Run keypoint annotator on one object of LineMod.
Type the following to see the meaning of options.
$ python annotate_keypoint.py --obj_id 1 --total_kp_number 50 --output_base ROOT/3_6Dpose_estimator/data --sixd_base DATAROOT
$ python annotate_keypoint.py -h
- The annotated keypoints are in file
annot_train.h5
andannot_eval.h5
. The corresponding training images are in folderstrain
andeval
.
- Relative files locate in
$ROOT/3_6Dpose_estimator/train_YOLO
.$ cd ROOT/3_6Dpose_estimator/train_YOLO
- Build Darknet (YOLOv3).
$ make
- Prepare data as AlexeyAB/darknet's instructions. Refer to folder
./scripts
for more help. - Download pretrained darknet53 here
- Run
train_single.sh
ortrain_all.sh
to train the network. - Put trained weights (e.g. 01.weights) in folder
$ROOT/3_6Dpose_estimator/models/yolo/
.
- Relative code is in
$ROOT/3_6Dpose_estimator/train_KPD/
$ cd ROOT/3_6Dpose_estimator/train_KPD
- Modify Line 19, 21, 39, 46 of file
./src/utils/dataset/coco.py
to previously annotated dataset. Examples are given in these lines. - Train on Linemod dataset without DPG.
$ python src/train.py --trainBatch 28 --expID seq5_Nov_1_1 --optMethod adam
- Train on Linemod dataset with DPG. Just add a
--addDPG
option. and load the model trained after in the second step.$ python src/train.py --trainBatch 28 --expID seq5_dpg_Nov_1_1 --optMethod adam --loadModel ./exp/coco/seq5_Nov_1_1/model_100.pkl --addDPG
- (Optional) Visualize training process. Type
$ tensorboard --logdir ./
-
Move back to the root of pose estimator.
$ cd ROOT/3_6Dpose_estimator/
-
Run the following command.
$ CUDA_VISIBLE_DEVICES=1 python3 betapose_evaluate.py --nClasses 50 --indir /01/eval --outdir examples/seq1 --sp --profile
The output json file containing predicted 6D poses will be in examples/seq1.