Mirgahney Mohamed · Lourdes Agapito

We propose DynamicSurf, a model-free neural implicit surface reconstruction method for high-fidelity 3D modelling of non-rigid surfaces from monocular RGB-D video.

The data is organized as NeuS

<case_name>
|-- cameras_sphere.npz    # camera parameters
|-- depth
    |-- # target depth for each view
    ...
|-- image
    |-- # target RGB each view
    ...
|-- mask
    |-- # target mask each view (For unmasked setting, set all pixels as 255)
    ...

Here cameras_sphere.npz follows the data format in IDR, where world_mat_xx denotes the world-to-image projection matrix, and scale_mat_xx denotes the normalization matrix.

You can download a part of pre-processed KillingFusion data here and unzip it into ./.

Important Tips: If the pre-processed data is useful, please cite the related paper(s) and strictly abide by related open-source license(s).

Clone this repository and create the environment (please notice CUDA version)

git clone https://github.com/Mirgahney/dysurf.git
cd dysurf

The code is tested with Python 3.9 and PyTorch 1.11 with CUDA 11.3. DynamicSurf requires smooth_sampler, which is a drop-in replacement for PyTorch's grid sampler that support double back-propagation. Also the following packages are required:

conda env create -f dysurf.yml
conda activate dysurf

• Training

python3 -W ignore exp_runner_grid_hydra.py 

• Evaluating pre-trained model

  Coming Soon

Coming Soon

Coming Soon

• Code of Data Pre-processing
• Code of Geometric Projection
• Pre-trained Models and Evaluation Code
• Training Code

This project is built upon NDR, Go-Surf, and NeuS. Some code snippets are also borrowed from IDR and NeRF-pytorch. The pre-processing code for camera pose initialization is borrowed from Fast-Robust-ICP. The evaluation code for geometry rendering is borrowed from StereoPIFu_Code. Thanks for these great projects. We thank all the authors for their great work and repos.

If you have questions, please contact Mirgahney Mohamed.

If you find our code or paper useful, please cite

@misc{mohamed2023dynamicsurf,
      title={DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid},
      author={Mirgahney Mohamed and Lourdes Agapito},
      year={2023},
      eprint={2311.08159},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

If you find our pre-processed data useful, please cite the related paper(s) and strictly abide by related open-source license(s).