REDNet
implementation of REDNet
“A Novel Recurrent Encoder-Decoder Structure for Large-Scale Multi-view Stereo Reconstruction from An Open Aerial Dataset”(arxiv:https://arxiv.org/abs/2003.00637)
The proposed network was trained and tested on a single NVIDIA TITAN RTX 2080Ti (24G).
Requirements
CUDA 10.0
cudnn 7.6
python 3.6
tensorflow-gpu 1.13.1
numpy 1.18
opencv-python 4.1
Data Preparation
- Download the WHU MVS dataset. http://gpcv.whu.edu.cn/data/WHU_dataset/WHU_MVS_dataset.zip.
- Unzip the dataset to the
WHU_MVS_dataset
folder.
Train
- In
train.py
, setdata_root
to your train data pathYOUR_PATH/WHU_MVS_dataset/train
- Train REDNet (TITAN RTX 2080Ti 24G):
python train.py
Test
- Download the pre-trained REDNet model (http://gpcv.whu.edu.cn/data/download/MODEL_FOLDER.zip)
Unzip it toMODEL_FOLDER
folder. - In
test.py
, setdense_folder
to your test data pathYOUR_PATH/WHU_MVS_dataset/test
, setmodel_dir
to your model pathMODEL_FOLDER
, set depth sample numbermax_d
. - Run REDNet:
python test.py
The test outputs are stored in YOUR_PATH/WHU_MVS_dataset/test/depths_rednet/
, including depth map XXX_init.pfm
, probability map XXX_prob.pfm
, scaled images XXX.jpg
and camera parameters XXX.txt
.
Predict
If you want to apply REDNet to your own data, please structure your data into such a folder.
Image Files
All image files are stored in the Images
folder.
Camera files
All camera files are stored in the Cams
folder.
The text file contains the camera extrinsic, intrinsic and the depth range:
extrinsic
E00 E01 E02 E03
E10 E11 E12 E13
E20 E21 E22 E23
E30 E31 E32 E33
f(pixel) x0(pixel) y0(pixel)
DEPTH_MIN DEPTH_MAX DEPTH_INTERVAL
IMAGE_INDEX 0 0 0 0 WIDTH HEIGHT
Make sure the camera extrinsic is Twc [Rwc|twc]
, camera orientation is XrightYup
.
- In
viewselection.py
, setdense_folder
to your data path. - Run:
python viewselection.py
The output file viewpair.txt
is stored in your data path.
- In
predict.py
, setdense_folder
to your data path. Setmodel_dir
to your model pathMODEL_FOLDER
, set depth sample numbermax_d
and image sizemax_w
,max_h
. - Run:
python predict.py
The outputs are stored in YOUR_DATA_PATH/depths_rednet/
.
We provided the script fusion.py
to apply depth map filter for post-processing, which converted the per-view depth maps to 3D point cloud.
- In
fusion.py
, setdense_folder
to your data path. - Run:
python fusion.py
Final point clouds are stored in YOUR_DATA_PATH/rednet_fusion/
.
Reference
This project is based on the implementation of R-MVSNet
. Thank Yao, Yao for providing the source code (https://github.com/YoYo000/MVSNet)