dps-net's Introduction

DPS-Net: Deep Polarimetric Stereo Depth Estimation

This repository is the official PyTorch implementation of the method present in

DPS-Net: Deep Polarimetric Stereo Depth Estimation

Chaoran Tian, Weihong Pan, Zimo Wang, Mao Mao, Guofeng Zhang, Hujun Bao, Ping Tan, Zhaopeng Cui

This repository is based on the Lipson's implementation of RAFT-Stereo. It is trained and tested in Ubuntu 20.04 + PyTorch 1.10.2 + RTX 3090.

Install

git clone https://github.com/Ethereal-Tiansss/DPS-Net.git
cd DPS-Net
conda env create -f environment.yml
conda activate dpsnet

Run

Data Preparation

Synthetic Data

Please follow IPS-Generator to synthetic polarimetric stereo dataset named as IPS dataset in our paper.

Real Data

The real polarimetric dataset is provide as well. The RPS dataset utilized in DPS-Net can be download from Google Drive.

Training

For convenience, we encapsulate all training and finetuning commands in scripts/train_<dataset>.sh To train our model, simply run:

python ./cmd/train_ips.sh
python ./cmd/train_rps.sh

Evaluation

To evaluate a trained model on a test set, run

python ./cmd/evaluate_ips.sh
python ./cmd/evaluate_rps.sh

Citing

We will appreciate it if you would like to cite our work via:

@inproceedings{tian2023dps,
  title={DPS-Net: Deep polarimetric stereo depth estimation},
  author={Tian, Chaoran and Pan, Weihong and Wang, Zimo and Mao, Mao and Zhang, Guofeng and Bao, Hujun and Tan, Ping and Cui, Zhaopeng},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={3569--3579},
  year={2023}
}

Acknowledgement

Thanks RAFT-Stereo, for providing nice and inspiring implementations of RAFT-Stereo. Thanks IRS for the open source stereo dataset, which includes the accurate surface normal and depth.

dps-net's People

Contributors

Stargazers

Watchers

dps-net's Issues

gradient computation has been modified by an inplace operation

when I run python ./cmd/train_rps.sh , get Error: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [6, 2, 40, 90]] is at version 9; expected version 7 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

This Error is positioned on line 220, in train.py -> scaler.scale(loss).backward().

I couldn't find the location for the 'inplace operation'. Can someone help me？thanks

What are the input polarizing images in dataset?

Your work is really great! However, I encounter some problems while reproducing the experiment. I got a really good result using the dataset you provided while get a bad result using my own dataset. I thought it could be wrong input of network. In your paper, the input of network are 6 images, 2 RGB image and 4 polarizing images(I guess they are aolp images and dolp images?). But in your code, you only input 4 images, 2 RGB images and 2 polarizing images. The polarizing image in your dataset is like below:

Is this image a combination of aolp image and dolp image? I will really appreciated it if you can solve my problem.

Recommend Projects

ethereal-tian / dps-net Goto Github PK