The code for our paper submitted to TIE: SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data.

SeqOT is a sequence-enhanced LiDAR-based place recognition method based on our previous work OverlapTransformer (OT) for RAL/IROS 2022.

Developed by Junyi Ma and Xieyuanli Chen.

Fig. 1 System overview with Haomo dataset and visualized evaluation on NCLT dataset. It can be seen that SeqOT is robust to driving directions.

1. Publications
2. Dependencies
3. How to use
4. Data preparation
5. License

The arXiv version is coming soon ...

We use pytorch-gpu for neural networks.

An nvidia GPU is needed for faster retrival.

To use a GPU, first you need to install the nvidia driver and CUDA.

• CUDA Installation guide: link
  We use CUDA 11.3 in our work. Other versions of CUDA are also supported but you should choose the corresponding torch version in the following Torch dependences.

• System dependencies:

  sudo apt-get update 
  sudo apt-get install -y python3-pip python3-tk
  sudo -H pip3 install --upgrade pip

• Torch dependences:
  Following this link, you can download Torch dependences by pip:

  pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

  or by conda:

  conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch

• Other Python dependencies (may also work with different versions than mentioned in the requirements file):

  sudo -H pip3 install -r requirements.txt

We provide a training and test tutorial for NCLT sequences in this repository.

You can start the two-step training with

cd train
python training_seqot.py 
python gen_sub_descriptors.py
python training_gem.py 

We will release our pretrained model after the paper is accepted.

The sub-descriptors of the database and query sequence have already been generated by python gen_sub_descriptors.py above. Here you can test SeqOT by

cd test
python test_gem_prepare.py
python test_seqot.py

Before training and test, please modify the params in the config.yml

• range_image_database_root: path of reference range images in database sequence.
• range_image_query_root: path of query range images in query sequence.

• traindata_file: index for training.
• poses_file: poses of the trajectory of the database.
• height: height of range images.
• width: width of range images.
• seqlen: sequence length to generate sub-descriptors.
• lr: learning rate for training feature extracter.
• resume: whether resuming from the pretrained model below.
• weights: pretrained feature extracter.

• seqlen: sequence length to generate sub-descriptors.
• weights: pretrained feature extracter.

• traindata_file: index for training (same as the one of training_seqot).
• poses_file: poses of the trajectory of the database (same as the one of training_seqot).
• descriptors_file: sub-descriptors of database for training (generated by gen_sub_descriptors.py).
• seqlen: sequence length to generate final-descriptors.
• lr: learning rate for training the pooling module.
• resume: whether resuming from the pretrained model below.
• weights: pretrained pooling module.

• sub_descriptors_database_file: sub-descriptors of database scans (generated by gen_sub_descriptors.py).
• sub_descriptors_query_file: sub-descriptors of query scans (generated by gen_sub_descriptors.py).
• seqlen: sequence length to generate final-descriptors (same as the one of training_gem).
• weights: pretrained pooling module.

groud_truth_file: ground truth containing correct loops.

• poses_database: poses of the trajectory of the database.
• poses_query: poses of the trajectory of the query sequence.

Please use the following commands to prepare data. We will also upload the preprocessed data used by SeqOT after the paper is accepted.

Please use this script to generate range images of NCLT dataset. You need to modify the params including scan_folder and dst_folder.

Please use this script to generate training indices of NCLT dataset. You need to modify the params including poses_database, poses_query, and scan_database_root.

cd data_prepararion
python ./gen_training_index.py 

Please use this script to generate groud truth loops of NCLT dataset.

cd data_prepararion
python ./gen_ground_truth.py 

Copyright 2022, Junyi Ma, Xieyuanli Chen, Jingyi Xu, Guangming Xiong, Beijing Institute of Technology.

This project is free software made available under the MIT License. For more details see the LICENSE file.