Dataset

• Dense Depth for Autonomous Driving (DDAD)
• KITTI Eigen Split

wget -i splits/kitti_archieves_to_download.txt -P kitti_data/
cd kitti_data/
unzip "*.zip"
cd ..
find kitti_data/ -name '*.png' | parallel 'convert -quality 92 -sampling-factor 2x2, 1x1, 1x1 {.}.png {.}.jpg && rm {}'

• The above conversion command creates images with default chroma subsampling 2x2, 1x1, 1x1.

Problem Setting

while specialist hardware can give per-pixel depth, a more attractive approach is to only require a single RGB camera.

train a deep network to map from an input image to a depth map

Methods

• Geometry Models

  The simplest representation of a camera an image plane at a given position and orientation in space.

  

  The pinhole camera geometry models the camera with two sub-parameterizations, intrinsic and extrinsic paramters. Intrinsic parameters model the optic component (without distortion), and extrinsic model the camera position and orientation in space. This projection of the camera is described as:

  

  A 3D point is projected in a image with the following formula (homogeneous coordinates):

  

• Cross-View Reconstruction

frames the learning problem as one of novel view-synthesis, by training a network to predict the appearance of a target image from the viewpoint another image using depth (disparity)

formulate the problem as the minimization of a photometric reprojection error at training time

Here. pe is a photometric reconstruction error, proj() are the resulting 2D coordinates of the projected depths Dₜ in the source view and <> is the sampling operator. For simplicity of notation we assume the pre-comuted intrinsics K of all views are identical, though they can be different. α is set to 0.85.

consider the scene structure and camera motion at the same time, where camera pose estimation has a positive impact on monocular depth estimation. these two sub-networks are trained jointly, and the entire model is constrained by image reconstruction loss similar to stereo matching methods. formulate the problem as the minimization of a photometric reprojection error at training time formulate the problem as the minimization of a photometric reprojection error at training time

Folder