LET-NET implements an extremely lightweight network for feature point extraction and image consistency computation. The network can process a 240 x 320 image on a CPU in about 5ms. Combined with LK optical flow, it breaks the assumption of brightness consistency and performs well on dynamic lighting as well as blurred images.
- The LET-NET training code is released at https://github.com/linyicheng1/LET-NET-Train.
- Gray Image is also suport in LET-NET, you can get pytorch and onnx model tpye in
./model/
- LET-VINS Demo run on UMA-VI dataset is released.
- Our proposed LET-VINS won the second place in the VIO track of the ICCV2023SLAM Challenge, which is the best performance among the traditional methods.
- The preprinted paper was posted at here.
![](https://private-user-images.githubusercontent.com/50650063/277204080-f4e96620-925e-4ff8-a2a9-c8af202c3b99.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjIwODExNzUsIm5iZiI6MTcyMjA4MDg3NSwicGF0aCI6Ii81MDY1MDA2My8yNzcyMDQwODAtZjRlOTY2MjAtOTI1ZS00ZmY4LWEyYTktYzhhZjIwMmMzYjk5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MjclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzI3VDExNDc1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTkwYmE3NjBlNjgwZGM3ZGJkNmJmZjJiOTExYTI1ZDRmOGFiZTJkYTVmNWUzMjNiOGJhZDgxNDhkOTk4NmRiZTMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.dlG5fslAhoq6Vwk9GKZXxRda0ja7E6YyLiUpVkp5RvA)
- Breaking of brightness consistency in optical flow with a lightweight CNN network,Yicheng Lin, Shuo Wang, Yunlong Jiang, Bin Han, arXiv:2310.15655, pdf
- OpenCV (https://docs.opencv.org/3.4/d7/d9f/tutorial_linux_install.html)
- ncnn (https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux)
Notes: After installing ncnn, you need to change the path in CMakeLists.txt
set(ncnn_DIR "<your_path>/install/lib/cmake/ncnn" CACHE PATH "Directory that contains ncnnConfig.cmake")
mkdir build && cd build
cmake .. && make -j4
You can enter the path to a video or two images.
./build/demo <path_param> <path_bin> <path_video>
or
./build/demo <path_param> <path_bin> <path_img_1> <path_img_2>
For example using the data we provide:
./build/demo ./model/model.param ./model/model.bin ./assets/nyu_snippet.mp4
You should see the following output from the NYU sequence snippet:
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
@ARTICLE{let-net,
author={Lin, Yicheng and Wang, Shuo and Jiang, Yunlong and Han, Bin},
journal={IEEE Robotics and Automation Letters},
title={{Breaking of brightness consistency in optical flow with a lightweight CNN network}},
year={2024},
pages={1-8}
}