Git Product home page Git Product logo

dit's Introduction

Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction

DIT-architecture

This repository contains python scripts for training and testing [Deep Interaction Transformer (DIT)]

Deep Interaction Transformer (DIT) is a full Transformer framework for point cloud registration, which achieves superior performance compared with current state-of-the-art learning-based methods in accuracy and robustness. DIT consists of the following three main modules:

  • a Point Cloud Structure Extractor for modeling global relation.
  • a Point Feature Transformer for improving the discrimination of features.
  • a GMCCE for correspondence confidence evaluation.

Configuration

This code is based on PyTorch implementation, and tested on:

  • Ubuntu 18.04
  • CUDA 11.1
  • pytorch 1.8.1
  • python 3.7.10

You can install the python requirements on your system with:

cd DIT
pip install -r requirements.txt

Training

You can run the relevant commands under the /DIT path to train the DIT model in a specific environment, the network parameters will be saved in the /models folder. Specifically, we use the ModelNet40 dataset for this work, which will be automatically downloaded if necessary.

Train the DIT on the clean, low noise partial, high noise partial point clouds as

sh experiments/1_train_clean.sh
sh experiments/1_train_low_noise_partial.sh
sh experiments/1_train_high_noise_partial.sh

Evaluation

We provide

  • pretrained models on ModelNet40 on clean, low noise partial, high noise partial point clouds. You can download it from this link weight. Unzip and place it in the /DIT folder, such that there are three pretrained models under the /DIT/models path;
  • three files 1_eval_clean.sh, 1_eval_low_noise_partial.sh, 1_eval_high_noise_partial.sh to evaluate the DIT model on clean, low noise partial, high noise partial point clouds in /experiments folder;
  • three files 1_eval_clean_vis.sh, 1_eval_low_noise_partial_vis.sh, 1_eval_high_noise_partial_vis.sh to visualize the DIT registration process on clean, low noise partial, high noise partial point clouds in /experiments folder;

You can run the relevant commands under the /DIT path to evaluate the DIT model in a specific environment. If you want to evaluate your training results, you can change the model path in the sh file directly

Evaluate the DIT on the clean, low noise partial, high noise partial point clouds as

sh experiments/1_eval_clean.sh
sh experiments/1_eval_low_noise_partial.sh
sh experiments/1_eval_high_noise_partial.sh

You can run the relevant commands under the /DIT path to visualize the DIT registration process in a specific environment as

sh experiments/1_eval_clean_vis.sh
sh experiments/1_eval_low_noise_partial_vis.sh
sh experiments/1_eval_high_noise_partial_vis.sh

dit's People

Contributors

cguangyan-bit avatar yuanli2333 avatar

Stargazers

 avatar  avatar  avatar Zihao Wang avatar hiyyg avatar Walter avatar  avatar  avatar TieNongZhang avatar J.K. Xia avatar Nishiwaki Soki avatar  avatar  avatar Yang Ai avatar Fu Lian avatar  avatar Nick Stephens avatar ashish hazara  avatar 爱可可-爱生活 avatar  avatar Jiang Zihang avatar Yujun Shi avatar Yongxing Dai avatar Matt Shaffer avatar  avatar

Watchers

Yongxing Dai avatar hiyyg avatar Matt Shaffer avatar  avatar

Forkers

silvesteryu

dit's Issues

Questions about training performance with other partially overlapping data

Thank you for doing such a great job. I noticed that you followed prnet's protocol for partially overlapping modelnet40 registration dataset and then I want to follow rpmnet's protocol to get partially overlapping modelnet40 dataset but I found that the training does not converge. The network has 60M size number of parameters, so I think I don't need to adjust the network, but I may need to adjust other hyper parameters. Looking forward to your reply, thanks.

UserWarning

Hello, why did I run your code with this warning from Userwarning? Is it an environmental issue or something else? Also, how did you adjust your model? On modelnet40, your model has high accuracy
IMG_20230716_095236

Is it suitable for other datasets?

Thanks for your great work!It is an inspiring paper.
But it is not experimented on the registration dataset 3DMatch/3DLoMatch.

I noticed that there is no downsampling process in the feature extraction stage,I wonder if DIT is not suitable for registration tasks with low overlap rate because of this problem.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.