News

• 04/04/2023 dataset preview release: 2 synthetic scenes available
• 15/04/2023 code release: 3D reconstruction and novel view synthesis part

TODO

• Full dataset release
• Code release for 3D reconstruction and novel view synthesis
• Code release for intrinsic decomposition and scene editing

conda env create -f environment.yml
conda activate i2sdf

Download our synthetic dataset and extract them into data/synthetic. If you want to run on your customized dataset, we provide a brief introduction to our data convention here.

We provide a high-quality synthetic indoor scene multi-view dataset, with ground truth camera pose and geometry annotations. See HERE for data conventions. Click HERE to download.

python main_recon.py --conf config/<config_file>.yml --scan_id <scan_id> -d <gpu_id> -v <version>

Note: config/synthetic.yml doesn't contain light mask network, while config/synthetic_light.yml contains.

python main_recon.py --conf config/<config_file>.yml --scan_id <scan_id> -d <gpu_id> -v <version> --test [--is_val] [--full]

The optional flag --is_val evaluates on the validation set instead of training set, --full produces full-resolution rendered images without downsampling.

python main_recon.py --conf config/<config_file>.yml --scan_id <scan_id> -d <gpu_id> -v <version> --test --test_mode interpolate --inter_id <view_id_0> <view_id_1> [--full]

Generates a view interpolation video between 2 views. Requires ffmpeg being installed.

The number of frames and frame rate of the video can be specified by options.

python main_recon.py --conf config/<config_file>.yml --scan_id <scan_id> -d <gpu_id> -v <version> --test --test_mode mesh

Brewing🍺, code coming soon.

If you find our work is useful, please consider cite:

@inproceedings{zhu2023i2sdf,
    title = {I$^2$-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs},
    author = {Jingsen Zhu and Yuchi Huo and Qi Ye and Fujun Luan and Jifan Li and Dianbing Xi and Lisha Wang and Rui Tang and Wei Hua and Hujun Bao and Rui Wang},
    booktitle = {CVPR},
    year = {2023}
}

• This repository is built upon Pytorch lightning.
• Thanks to Lior Yariv for her excellent work VolSDF.
• Thanks to Scalable-NISR team for providing their real-world dataset.