sraingit / cae-lo Goto Github PK

CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description

Python 80.05% M 0.11% MATLAB 19.85%

lidar-odometry deep-learning cnn auto-encoder point-cloud interest-points feature-extraction

cae-lo's Introduction

CAE-LO

CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description

@article{yin2020caelo,
    title={CAE-LO: LiDAR Odometry Leveraging Fully Unsupervised Convolutional Auto-Encoder for Interest Point Detection and Feature Description},
    author={Deyu Yin and Qian Zhang and Jingbin Liu and Xinlian Liang and Yunsheng Wang and Jyri Maanpää and Hao Ma and Juha Hyyppä and Ruizhi Chen},
    journal={arXiv preprint arXiv:2001.01354},
    year={2020}
}
@article{
    title={Interest Point Detection from Multi-Beam LiDAR Point Cloud Using Unsupervised CNN},
    author={Deyu Yin, Qian Zhang, Jingbin Liu, Xinlian Liang, Yunsheng Wang, Shoubin Chen, Jyri Maanpää, Juha Hyyppä, Ruizhi Chen},
    journal={IET Image Processing},
    year={2020}
}

See the rankings in KITTI, our method's name is "CAE-LO".

Usage

Basic enviornments for python3 and Keras. Simple networks. No worries. Package requirements can be found in requirements.txt.
Dirs.py to modify dictionaries.
BatchProcess.py projects PCs to spherical rings with multi-thread processing.
BatchVoxelization.py project PCs into multi-solution voxel model with multi-thread processing.
SphericalRing.py to do basic function about spherical ring model, especially the function of getting keypts.
You can try Match.py to see some demos using trained models.
PoseEstimation.py to generate initial odometry.
RefinePoses.py to generate refined odometry based on extended interest points & ground normals, and also to show generated trajectories. (The code for generating ground normals is currently commented. Uncomment it if you want to use.)

Notes

Generated interest points and features for sequence 00 and 01 can be found in GoogleDrive.
The final refined trajectories of sequence 00-10 can be found in GoogleDrive.
The data arragement format is simple. Just folders like "KeyPts", "Features", "InliersIdx", "SphericalRing", etc.
Package PCLKeypoint in PclKeyPts.py can be installed from: https://github.com/lijx10/PCLKeypoints.
If you have any problems or confunsions, please post them in ISSUES or contact me by email.

cae-lo's People

Contributors

Stargazers

Watchers

cae-lo's Issues

Demo data does not contain ground truth poses

I have 2 question:

Can i get a snippet of the folder structure for the project. Maybe a tree format of it?
The DemoData does not contain all the snippet of the data required to run the code.
Can I get the data for poses, poses__, poses___ ? and any other sample data you can share.

Thanks in advance

Prediction of LiDAR pointcloud odometry

I wish to give as input a sequence of point cloud data to your AI model and then the model outputs a sequence of final odometry values (in a form rotation and translation).
How can I achieve this ?

Thanks in advance.

May i also get your email address?

Training ,Model and inference code

@SRainGit hi thanks for wonderful work , is it possible to share the pre-trained model with training and inference pipeline
THnaks in advance

About Interest points and features

Hello, this project is really awesome. However, I cannot load keypoints and featured because there is no Baidu account. Can you help me by emailing me? Thank you very much.

what is python version

Question about the pose refinement.

Hi, Thanks for sharing. I have some questions regards the pose refinement step. In the paper, you claimed that it is based on ICP and you only use a smaller number of EIPs to reduce the computation cost. In my view, the reason why ICP can achieve highly accurate results is that it uses dense point cloud to build the correspondences, while the initial odometry got by RANSAC is estimated using sparse keypoint correspondences. So I am really confused about this step. How many EIPs you are using for ICP? And have you done some ablation studies on the number of EIPs vs. the error decrease?

Thank you for your time.

How to plot 3d pointcloud feature matching like figure 6 in your paper?

Hi，
I want to draw the line between the 3d key points to represent the matching relationship and I found figure 6 in your paper very awesome, So How to plot 3d pointcloud feature matching like that?

Thanks~

A question about the detail when pre-process KITTI Velodyne data

Hi~
Thanks for your great work.

I am quite new with KITTI Velodyne and confused about this "error" :

an error of 0.22° in the Velodyne sensor

Does this angle error occur in all KITTI Odometry training sequences (00-10)?
I also read the relevant paper[46], but still don't know how to correct it...

Can you explain more in detail?

Thanks in advance~

GetKeyPtsAndPatchesFromKeyPts function cannot be found in RefinePoses.py

I cannot find GetKeyPtsAndPatchesFromKeyPts functionality that is stated in the RefinePose.py

Please help me add this function or locate it. Thanks in advance

Which data can be substituted for dirKeyPts3DFeatNet in PoseEstimation.py?

In the PoseEstimation.py file on line 177 :
dirKeyPts3DFeatNet = '/media/rain/Win10_F/KITTI_odometry/output_3DFeatNet/Descriptors/'

Which data can I substitute for the dirKeyPts3DFeatNet assuming I am training it on a different data set ?

Also can you please share the features data as well?

Thanks a lot for your help.

PlanarPts from the keyPtsData is empty

I realized that the variable PlanarPts is initialized as an empty np.array(), these empty array values are stored in the keypts (*.bin.mat) files.

How can I populate this values stored in the keypts files because it is needed in the refinePoses.py, here:
PlanarPts1_ = copy.deepcopy(PlanarPts1)
PlanarPts1_[:, 0:3] = np.array(((np.dot(oriRelR, PlanarPts1[:, 0:3].T) + oriRelT).T), dtype=np.float32)

How can I resolve this?