Author: Luigi Freda
slamplay is a collection of powerful tools to start playing and experimenting with SLAM in C++. It's a work in progress. It installs and makes available in a single cmake framework some of the most important
- back-end frameworks (g2o, gtsam, ceres, se-sync, etc.),
- front-end tools (OpenCV, pcl, TensorRT, tensorflow_cc, etc.),
- algebra and geometry libs (eigen, sophus, cholmod, etc.),
- viz tools (pangolin, imgui, rerun, etc.),
- loop-closure frameworks (DBOW3, iBOW, etc.),
- deep learning tools (based on TensorRT, tensorflow_cc, etc.),
along with some nice examples in order to easily and quickly start with all these tools.
I created slamplay for a computer vision class I recently taught. I started developing it for fun, during my free time, taking inspiration from some repos available on the web.
This repository is structured in the following main folders (with self-explanatory names):
algebra_geometry
backend
data
dense_mapping
docs
frontend
(with new C++ tools based on TensorRT and tensorflow_cc, e.g, SuperPoint, SuperGlue, Depth-Anything, HFNet)full_slam
io
loop_closure
dense_mapping
scripts
utils
viz
The following procedure has been successfully tested under Ubuntu 20.04 (and under Ubuntu 22.04 with partial DL support).
- Install basic dependencies:
$ ./install_dependencies.sh
- Install OpenCV in a local folder:
$ ./install_local_opencv.sh
(if you want, skip this step and set the variableOpenCV_DIR
inconfig.sh
with your local OpenCV path) - Build the framework:
$ ./build.sh
Once everything is built, you can enter in the build
folder and test the different examples.
In particular, to test the full_slam
apps:
- Configure
full_slam/config/kitti.yaml
(orfull_slam/config/euroc.yaml
) - Then run the VO (Visual Odometry) app:
$ cd build/full_slam/apps $ ./run_kitti_stereo # or ./run_euroc_stereo
Additional steps:
- If you want to smoothly test the examples then dowload the provided testing images and videos (will be deployed in
data
folder):
$ ./install_data.sh
- If you want to use the DL (Deep Learning) models then run the following command to download both NN weights/checkpoints with their related data:
$ ./install_dl_models.sh
- If you want to install and test tensorflow C++ API (e.g. for HFNet), then run:
$ ./install_tensorflow_cc.sh
See tensorflow_cc for further details. Note that this step will take a long while.
See the nice ascii quick reference.
- Quick reference
- Dense matrix manipulation
- Dense linear problems and decompositions
- Sparse linear algebra
- Geometry
A couple of notes about some of the available front-end features.
In frontend/feature_dl
you can find:
- A C++ implementation of SuperPoint and SuperGlue under TensorRT
- A C++ implementation of HFNet under TensorRT and Tensorflow.
In frontend/depth_dl
you can find:
- A C++ implementation of Depth-Anything-V2.0 based on TensorRT.
As explained above, if you want to install and test tensorflow C++ API (e.g. for HFNet), then run:
$ ./install_tensorflow_cc.sh
See tensorflow_cc for further details. Note that this step will take a long while.
My current preferred working configuration (default one) under Ubuntu 20.04:
- C++: 17
- TENSORFLOW_VERSION: 2.9.0
- BAZEL_VERSION: 5.1.1
- CUDA: 11.6
- CUDNN: 8.6.0.163-1+cuda11.8
sudo apt install -y libcudnn8=8.6.0.163-1+cuda11.8
sudo apt install -y libcudnn8-dev=8.6.0.163-1+cuda11.8
A couple of notes about some of the available back-end frameworks.
Installed tag 4.2a9
Documentation
- https://gtsam.org/docs/
- https://gtsam.org/tutorials/intro.html
- See
docs
folder for further documentation.
ISSUES
Apparently, in order to avoid double free or corruption errors with gtsam on exit, we need to disable the compile option -march=native
when building apps that use gtsam. This can be done locally by modifying the compile flags at the folder level and removing march native optimization for gtsam-related files. Further details are in the following links:
- https://bitbucket.org/gtborg/gtsam/issues/414/compiling-with-march-native-results-in
- https://groups.google.com/g/gtsam-users/c/jdySXchYVQg
Installed tag 2.1.0
Documentation
- http://ceres-solver.org/tutorial.html
- See
docs
folder for further documentation.
Installed tag 20230223_git. See related examples.
Issues:
- The built g2o-dependant binaries link to the system g2o (instead of the locally compiled and installed g2o) and this brings to crashes (a different g2o version is linked). There are different solutions to this problem:
- Standard one (uncomfortable): use
LD_LIBRARY_PATH
to make the built binaries correctly link to the locally compiled and installed g2o. - Otherwise, we can use and set
RPATH
(instead ofRUNPATH
) at build time. In particular, this can be done by using some compiler options. This is what I set in my cmake configuration:set(MY_FLAGS "${MY_FLAGS} -Wl,--disable-new-dtags")
https://stackoverflow.com/questions/47117443/dynamic-linking-with-rpath-not-working-under-ubuntu-17-10 This configuration is enabled/disabled by the cmake option flagSET_RPATH
I added.
- Standard one (uncomfortable): use
- If you get a double free or corruption error with g2o (on exit), then it is very likely you used
-march=native
option when compiling this project but you didn't use the same option for building g2o itself. This may cause some alignment inconsistencies between g2o and this project. Then, in that case, build g2o with-march=native
(i.e. use the cmake option-DBUILD_WITH_MARCH_NATIVE=ON
)
A couple of notes about the IO library utils.
https://www.modernescpp.com/index.php/the-three-clocks
Differences amongst the three different clocks?
- std::chrono::sytem_clock: This is the system-wide real-time clock (wall-clock). The clock has the auxiliary functions to_time_t and from_time_t to convert time points into dates.
- std::chrono::steady_clock: Provides as only a clock the guarantee that you can not adjust it. Therefore, std::chrono::steady_clock is the preferred clock to wait for a time duration or until a time point.
- std::chrono::high_resolution_clock: This is the clock with the highest accuracy, but it can be a synonym for the clock's std::chrono::system_clock or std::chrono::steady_clock.
The C++ standard provides no guarantee about the clocks' accuracy, starting point, or valid time range. Typically, the starting point of std::chrono:system_clock is the 1.1.1970, the so-called UNIX-epoch. For std::chrono::steady_clock, typically the boot time of your PC.
What is the difference between steady_clock vs system_clock in layman terms?
If you're holding a system_clock in your hand, you would call it a watch, and it would tell you what time it is.
If you're holding a steady_clock in your hand, you would call it a stopwatch, and it would tell you how fast someone ran a lap, but it would not tell you what time it is.
Tracy is a great profiler. Repository link: https://github.com/wolfpld/tracy. Documentation is here.
- Include
profiler/profiler_tracy.h
in every file you are interested in profiling. - Check
TRACY_ENABLE
is defined for the WHOLE project (slamplay
automatically takes care of that if you setUSE_TRACY=1
inconfig.h
). - Add the macro
FrameMark
at the end of each frame loop. (Jump this: The latest version of tracy does not seem to support well this) - Add the macro
ZoneScoped
as the first line of your function definitions to include them in the profile. - Run tracy profiler:
./thirdparty/tracy/tracy-profiler
and hit the connect button. - Compile and run your application.
- You’re profiling your program! Go to tracy interface and hit the
statistics
button.
- This repo imported some of the C++ examples (updated, improved, and commented) of the repository https://github.com/gaoxiang12/slambook2. Thanks to the Author for his great work.
- Thanks to the Author of the repository https://github.com/nicolov/simple_slam_loop_closure/. I imported a couple of scripts (updated and improved) from his repository (for computing the confusion matrix).
- Thanks to yuefanhao for his repo https://github.com/yuefanhao/SuperPoint-SuperGlue-TensorRT
- Thanks to the Authors of the repository https://github.com/LiuLimingCode/HFNet_SLAM. I adapted their C++ implementation of HFNet NN.
- Thanks to the Authors of the repositories https://github.com/spacewalk01/depth-anything-tensorrt and https://github.com/ojh6404/depth_anything_ros. I adapted some of their C++ classes and script for DepthAnything v2 NN.