segstereo's Introduction

SegStereo

Caffe implementation of SegStereo and ResNetCorr models.

Requirements

This code is tested with Caffe, CUDA 8.0 and Ubuntu 16.04.

Basic caffe implementation is from Caffe.
The correlation and correlation1d layers are from FlowNet 2.0.
The Interp layer is from PSPNet.
The disparity tool is from OpticalFlowToolkit

Data

Our models require rectified stereo pairs. We provide several examples in data directory

Models

SegStereo_SRC_pretrain.caffemodel: Google Drive
ResNetCorr_SRC_pretrain.caffemodel: Google Drive
SegStereo_KITTI_finetune.caffemodel: Google Drive
SegStereo_pre_corr_finetune.caffemodel: Google Drive
ResNetCorr_KITTI_finetune.caffemodel: Google Drive

Evaluation

To test or evaluate the disparity model, you can use the script in model/get_disp.py. We recommend that you put the model under correponding directory.

python get_disp.py --model_weights ./ResNetCorr/ResNetCorr_SRC_pretrain.caffemodel --model_deploy ./ResNetCorr/ResNetCorr_deploy.prototxt --data ../data/KITTI --result ./ResNetCorr/result/kitti --gpu 0 --colorize --evaluate

Reference

If our SegStereo or ResNetCorr models help your research, please consider citing:

@inproceedings{yang2018SegStereo,
  author    = {Yang, Guorun and
               Zhao, Hengshuang and
               Shi, Jianping and
               Deng, Zhidong and
               Jia, Jiaya},
  title     = {SegStereo: Exploiting Semantic Information for Disparity Estimation},
  booktitle = ECCV,
  year      = {2018}
}

If you find our synthetic realistic collaborative (SRC) training strategy useful, please consider citing:

@inproceedings{yang2018srcdisp,
  author    = {Yang, Guorun and
               Deng, Zhidong and
               Lu, Hongchao and
               Li, Zeping},
  title     = {SRC-Disp: Synthetic-Realistic Collaborative Disparity Learning for Stereo Mathcing},
  booktitle = ACCV,
  year      = {2018}
}