PCD Reconstruction and Registration for the ML3D course at TUM (2022)
Finding the affine transformation between two point clouds is an ongoing research work in computer vision and has applications in computer graphics, robotics, etc. Several powerful algorithms have been proposed for this purpose, such as the Iterative closest-points (ICP). However, it has been shown that they tend to converge to poor optima or saddle points. This project investigats the use of a deep learning approach, Deep-Closest Point (DCP), to solve the problem. However, in their work, it was found that the algorithm has difficulty aligning incomplete point clouds with corresponding shapes. We found that a simple solution is to perform transfer learning by extending the training set to handle incomplete shapes. In addition, we performed a comparison of the use of DCP and ICP on incomplete shapes and found promising results.
- Create virtual environment via
conda
conda create -n [name] python=3.7
conda activate [name]
- Install pytorch
pip install torch==1.12.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
- Install requirements
pip install -r requirements.txt
- Install cuda
sh setup_env.sh
.
├── ...
├── Configs
│ └── pr_config.py # Configuration file with model parameters
├── DCP # Folder for the DCP model
├── docs # Folder with the final paper and other resources
├── docs # Folder with the trained models
├── SpareNet # Folder for the SpareNet model
└── README.md
Two different datasets were used:
- ShapeNet: download link (16384 points) generated by GRNet, and the KITTI dataset;
- ModelNet40; link
Final folder structure:
data
├── ModelNet40 # Original ModelNet DCP dataset
└── ShapeNetCompletion # ShapeNet dataset
To run the model simply configure the pr_config.py
file (see folder structure) and run the main script by calling on the root:
python main.py
You can select the model from the model
folder.
Outputs
├── Orig # Folder with outputs for the original DCP
│ ├── gtcloud # Folder with full point cloud to full point cloud matches
│ ├── partial_cloud # Folder with partial point cloud to full point cloud matches
│ └── reconstructed # Folder with reconstructed point cloud to full point cloud matches
└── Trained # Folder with outputs for the trained DCP
└── ... # Exact same folder structure as in Orig