heyijia / pl-vio Goto Github PK

monocular visual inertial system with point and line features

License: GNU General Public License v3.0

CMake 0.74% C++ 97.72% Python 1.54%

pl-vio's Introduction

PL-VIO

Tightly-Coupled Monocular Visual–Inertial Odometry Using Point and Line Features

Compared with point features, lines provide significantly more geometry structure information of the environment. We proposed PL-VIO a tightly-coupled monocular visual-inertial odometry system exploiting both point and line features. This code runs on Linux, and is fully integrated with ROS.

1. Prerequisites

1.1 Ubuntu and ROS Ubuntu 16.04. ROS Kinetic, ROS Installation additional ROS pacakge

	sudo apt-get install ros-YOUR_DISTRO-cv-bridge ros-YOUR_DISTRO-tf ros-YOUR_DISTRO-message-filters ros-YOUR_DISTRO-image-transport

If you install ROS Kinetic, please update opencv3 with

    sudo apt-get install ros-kinetic-opencv3

1.2. Ceres Solver Follow Ceres Installation, remember to make install.

2. Build PL-VIO on ROS

Clone the repository and catkin_make:

    cd ~/catkin_ws/src
    git clone https://github.com/HeYijia/PL-VIO.git
    cd ../
    catkin_make
    source ~/catkin_ws/devel/setup.bash

3. Run with Simulation Dataset

we have provide the simulation data, you can test plvio with it.

	roslaunch plvio_estimator simdata_fix.launch 
	roslaunch plvio_estimator vins_rviz.launch

green for trajectory, blue for point and line landmarks.

Notice: if you want to generate your own data, you can refer to the code below.

	https://github.com/HeYijia/vio_data_simulation

4.Performance on EuRoC dataset

4.1 Run with feature.bag

Since line detection and matching are time consuming, we can record the feature results from the front-end module as feature.bag and test PL-VIO's back-end module. You can find mh05_feature.bag in config fold and test our system with it:

	roslaunch plvio_estimator euroc_fix_offline.launch 
	roslaunch plvio_estimator vins_rviz.launch 
	rosbag play -r 5 config/mh05_feature.bag

4.2 Run with EuRoC dataset directly

4.2.1 Download EuRoC MAV Dataset. Although it contains stereo cameras, we only use one camera.

4.2.2 Open three terminals, launch the vins_estimator , rviz and play the bag file respectively. Take MH_05 as example

    roslaunch plvio_estimator euroc_fix_extrinsic.launch 
    roslaunch plvio_estimator vins_rviz.launch 
    rosbag play -r 0.2 YOUR_PATH_TO_DATASET/MH_05_difficult.bag

(If you fail to open vins_rviz.launch, just open an empty rviz, then load the config file: file -> Open Config-> YOUR_VINS_FOLDER/config/vins_rviz_config.rviz) **Notice: ** Please play your bag with 0.2x speed since the time consuming from line detection.

5 Related Papers

PL-VIO: Tightly-Coupled Monocular Visual–Inertial Odometry Using Point and Line Features, Yijia He, Ji Zhao, Yue Guo, Wenhao He and Kui Yuan.

@article{he2018pl,
  title={Pl-vio: Tightly-coupled monocular visual--inertial odometry using point and line features},
  author={He, Yijia and Zhao, Ji and Guo, Yue and He, Wenhao and Yuan, Kui},
  journal={Sensors},
  volume={18},
  number={4},
  pages={1159},
  year={2018},
  publisher={Multidisciplinary Digital Publishing Institute}
}

If you use PL-VIO for your academic research, please cite our related papers.

6. Acknowledgements

We use VINS-Mono as our base line code. Thanks Dr. Qin Tong, Prof. Shen etc very much.

7. Licence

The source code is released under GPLv3 license.

We are still working on improving the code reliability. For any technical issues, please contact Yijia He [email protected].

pl-vio's People

Contributors

Stargazers

Watchers

Forkers

feifanrensheng cxzhp lisilin013 maxee1900 arenas7307979 jiancui1992 simonsroad chnhs xiaranqing mrwangmaomao jczeng1212 hs5530hs joeyleenpu grandzxw wf-hahaha aaravrav142 euwen lyh-young li9616 calm-wy zbqq yoshua-msc-thesis min-zou moshanatucsd jiaowohuanshao yuyadanyadan wbercode liming-cheng kuankuan-yue tzhangzju nanlover hchrobo zhouhaoran-tju dongyanchaotj qianqii kintzhao nickxiang0306 zebrajack taihup mszy17 celesius match08 jarsam drscopus lulllabs wanghaoweiqing peter2527 kongan hansry daxiaoshuojia996 lhospitallky clssucceed marcelomata xinyangrobotic xwspace lixinhaiyk yongergui fgq1992 lijian8 yuantao15 chaochao92 yuangao2019 pangzh anyong298 hlm12348765 ohhandsome zoe4751 guoxiaoyang12 watcher-zy bobdeng1974 tanjiquan hzm8341 fbz1993 codergenj zeta1999 chengwei920412 wmh123456789 qq664956261 mfkiwl hitzhang boxulibrary zxawangxiaomu liaweicool cartesian123 fengpan1010 zhl8223 fightmoney ral-lee-ls 0iui0 cppc1024 roger-chuh wuxinxiaoshifu kailuodewangzi h-k8888 hjdkl99 maxibooksiyi ziyu1018 jackliuday tne-ai junyang224

pl-vio's Issues

run catkin_make

cmake error at pl-vins/benchmark_publisher/cmakelists.txt:18 (find_package): by not providing "findeigen3.cmake" in cmake_module_path this project has asked cmake to find a package configuration file provided by "eigen3", but cmake did not find one. could not find a package configuration file provided by "eigen3" with any of the following names: eigen3config.cmake eigen3-config.cmake

请问如何将3d map 转换成2d 栅格地图？

贺博，你好！有个问题需要咨询一下，请问如何将PL-VIO 生成的3d map 转换成2d 栅格地图？目的是借助2d栅格地图来进行导航

为什么我的rviz没有一点图像

Euroc数据集的V1_02导航结果漂移

想请问一下这个数据集上的结果怎么样才能不漂移呢？
看了一些论文，有些PLVIO的v102结果为×，表示跑不出来，部分的结果rmse为0.1m左右

怎么证明法向量夹角小的运动就是相机位姿变化呢

@HeYijia

还是说假设在帧间时间极短，两帧间角度拉开不了太大呢？
但是三角化不是一般要求有比较大的移动吗？

关于线特征边缘化的问题

为什么对线特征边缘化是采用 ResidualBlockInfo *residual_block_info = new ResidualBlockInfo(f, loss_function,vector<double *>{para_Pose[imu_j], para_Ex_Pose[0], para_LineFeature[linefeature_index]},drop_set); drop_set = vector{2}; 而不是像点特征一样边缘化滑窗中第0帧，drop_set = {0,2}

编译报错line_descriptor下几个函数是为定义的引用

贺博您好，今天是在ubuntu18.04，ROS-melodic下编译的PL-VIO一直报错说cv::line_descriptor::LSDDetector::createLSDDetector()，cv::line_descriptor::BinaryDescriptor::createBinaryDescriptor()等几个函数是未定义的引用，修改CMakeLists.txt把OpenCV那个头文件的绝对路径也加到里面还是报错（opencv版本是3.4.6)，希望贺博可以给我一些帮助。

catkin_make have error，but not have any red tips

Linking CXX executable /home/zhujie01/catkin_ws_pl/devel/lib/plvio_estimator/plvio_estimator
[ 95%] Built target plvio_estimator
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j12 -l12" failed

lineDetector类找不到

贺博你好，在linefeature_tracker.cpp文件的169行中，lineDetector类找不到，请问是没有上传吗？

支持鱼眼相机模型吗？

您好，贺博，我在使用鱼眼相机模型，发现线特征show窗口图像被放大了。

rviz配置文件

你好贺博，请问config/vins_rviz_config.rviz在哪里？我在config文件夹中没有找到响应的文件。

可以在ubuntu14.04 + ros-indigo 环境下编译吗？

如题
谢谢！

Run with my device？

Can i run this code with my device such as zed/zed which has camer and imu？
In my opinio，i just need to replace the launch file，when i launch my device with ros，the plvio system would work。
Appreciate it to get your advice！

关于恢复线的三维位置

visualization.cpp里面369行Lc和您论文里面关于Lc的公式似乎不一样？公式（14）

还有对之后恢复线端点3维位置不是很明白？

希望您能解答一下感谢！！

Lc << skew_symmetric(nc), vc, -vc.transpose(), 0;

Vector4d obs = it_per_id.linefeature_per_frame[0].lineobs;   // 第一次观测到这帧
Vector3d p11 = Vector3d(obs(0), obs(1), 1.0);
Vector3d p21 = Vector3d(obs(2), obs(3), 1.0);
Vector2d ln = ( p11.cross(p21) ).head(2);     // 直线的垂直方向
ln = ln / ln.norm();

Vector3d p12 = Vector3d(p11(0) + ln(0), p11(1) + ln(1), 1.0);  // 直线垂直方向上移动一个单位
Vector3d p22 = Vector3d(p21(0) + ln(0), p21(1) + ln(1), 1.0);
Vector3d cam = Vector3d( 0, 0, 0 );

Vector4d pi1 = pi_from_ppp(cam, p11, p12);
Vector4d pi2 = pi_from_ppp(cam, p21, p22);

Vector4d e1 = Lc * pi1;
Vector4d e2 = Lc * pi2;
e1 = e1/e1(3);
e2 = e2/e2(3);

Vector6d line_to_pose(Vector6d line_w, Eigen::Matrix3d Rcw, Eigen::Vector3d tcw) {
    Vector6d line_c;

    Vector3d cp_w, dv_w;
    cp_w = line_w.head(3);
    dv_w = line_w.tail(3);

    Vector3d cp_c = point_to_pose( Rcw, tcw, cp_w );
    Vector3d dv_c = Rcw* dv_w;

    line_c.head(3) = cp_c;
    line_c.tail(3) = dv_c;

    return line_c;
}

贺博好, 想问一下这里的 line 是用的什么形式呢? 看起来并不是 pluecker 坐标系, 因为与库中 plk_to_pose 描写的转换关系不太一样.

感谢~