News:

• Our CDPN wins "The Best RGB-Only Method Awards" on the BOP Challenge on ICCV2019 (code link: https://github.com/LZGMatrix/BOP19_CDPN_2019ICCV).
• Our CDPNv2 wins the "The Best Methods on Individual Datasets" on the BOP Challenge on ECCV2020 (code link: https://github.com/LZGMatrix/BOP19_CDPN_2019ICCV/tree/bop2020).

CDPN: Coordinates-based Disentangled Pose Network for Real-time RGB-based 6-DoF Object Pose Estimation

CDPN: Coordinates-based Disentangled Pose Network for Real-time RGB-based 6-DoF Object Pose Estimation
Zhigang Li, Gu Wang, Xiangyang Ji
ICCV 2019 (Oral) paper, supplement, oral

We provide the clean-version training code of our ICCV 2019 paper "CDPN: Coordinates-based Disentangled Pose Network for Real-time RGB-based 6-DoF Object Pose Estimation". This code can perfectly reproduce the impressive results of our paper.

If you find this code useful for your research, please cite our paper:

@InProceedings{Li_2019_ICCV,
author = {Li, Zhigang and Wang, Gu and Ji, Xiangyang},
title = {CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}

Overview

6-DoF object pose estimation from a single RGB image is a fundamental and long-standing problem in computer vision. Current leading approaches solve it by training deep networks to either regress both rotation and translation from image directly or to construct 2D-3D correspondences and further solve them via PnP indirectly. We argue that rotation and translation should be treated differently for their significant difference. In this work, we propose a novel 6-DoF pose estimation approach: Coordinates-based Disentangled Pose Network (CDPN), which disentangles the pose to predict rotation and translation separately to achieve highly accurate and robust pose estimation. Our method is flexible, efficient, highly accurate and can deal with texture-less and occluded objects. Extensive experiments on LINEMOD and Occlusion datasets are conducted and demonstrate the superiority of our approach. Concretely, our approach significantly exceeds the state-of-theart RGB-based methods on commonly used metrics.

Results in our paper

Environment

Set up python 3.6.7 environment：

pytorch==1.4.0
torchvision==0.5.0
numpy==1.19.3
opencv==4.5.0
tensorboardx==2.1

Other dependencies: 
yaml, pickle, pyparsing, progress, plyfile, scipy, tqdm, glob, os, sys...

Prepare the dataset

1. Download training and test data of LINEMOD dataset:

   • The OpenGL training data can be found here (password: b7kt).
   • The real training data can be found here (password: sesw).
   • The test data can be found here (password: mhko).
   • The 3D object models can be found here (password: ba4f).

2. Download VOC2012 dataset from official website (http://host.robots.ox.ac.uk/pascal/VOC/index.html).

   Note: The VOC2012 is only used as background data, which can also be replaced with other real-world datasets such as MS COCO dataset, SUN datset, and so on.

3. Prepare the dataset as follows:

Root
├── dataset
│   ├── bg_images
│   │   └── VOC2012
│   └── lm
│       ├── models
│       │   ├── ape
│       │   └── ...
│       ├── imgn
│       │   ├── ape
│       │   └── ...
│       ├── real_test
│       │   ├── ape
│       │   └── ...
│       └── real_train
│           ├── ape
│           └── ...
├── asserts
├── tools
├── lib
├── dataset_cache (automatically built in the first running)
└── exp (automatically built in running)

Training

1. Stage 1:

• cd tools; and run:

sh train_stage1.sh

2. Stage 2:

• Edit the config file of stage 2 in path ./tools/exps_cfg/config_trans.yaml:

  ​ Replace the 'path to the trained model of step1' with your actual path.

• cd tools; and run:

sh train_stage2.sh

3. Stage 3:

• Edit the config file of stage 3 in path ./tools/exps_cfg/config_rot_trans.yaml:

  ​ Replace the 'path to the trained model of step2' with your actual path.

• cd tools; and run:

sh train_stage3.sh

Testing

For testing various models trained in stage1, stage2 and stage3:

1. Edit the config file by:
   • Setting the "test" from False to True.
   • Setting the "load_model" with path of the testing model.
2. cd tools; and run:

python main.py --cfg=exps_cfg/path_to_your_config_file.yaml

Our trained models

Our trained model (download link, password: 92e2) and log file (download link, password: n792) of stage 1:

Our trained model (download link, password: q8rq) and log file (download, password: mdle) of stage 2:

Our trained model (download, password: ksoo) and log file (download, password: tb1q) of stage 3:

Acknowledgement

This work is affliated with Tsinghua University Xiangyang Ji's Lab. For business cooperation, welcome to contact: [email protected] & [email protected].

Copyright (c) Tsinghua University Xiangyang Ji's Lab. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.