Reproduce Visual Localization Work about loftr HOT 9 CLOSED

Hi zhongyuan, this is a very good question!
Inspired by the code from DualRCNet, we didn't use sub-pixel predictions for db-db pairs by using round(). Therefore the "keypoints" will be strictly repeatable across images. I'll show you the reconstruction of Aachen in the image below. And the errors are reduced by the triangulation process and a high image resolution. We use sub-pixel predictions for db-query pairs, which contributes to the high localization accuracy.

from loftr.

Thanks for quick reply. It seems like continuous keyframes can still produce similar keypoints distribution due to round() function.
Following the localization strategy, the standard flow is: image retrieval -> 2D-2D match -> 3D-2D pnp -> pose result. Owing to round() during structure from motion, 3D points are binding with integer keypoints in keyframes. So during localization, how to handle the correspondence between 3D points and the new sub-pixel db keypoints or maybe the sup-pixel strategy is only used in query images ?

from loftr.

Correct. For LoFTR, we use sub-pixel strategy only for query images currently.

from loftr.

Thanks a lot. I'll have a try!

from loftr.

No problem 👏

from loftr.

Hi @Zhongyuan-Li @zehongs ,
I have a question about this part
"we use sub-pixel strategy only for query images currently."

Could you explain this part in more detail?

from loftr.

Hi,
For visual localization, one needs to first build a map with database images, and then establish correspondences for database-query images. Specifically, to find correspondences of the database points and query points.
Here, we round the point location when building database map. And we use sub-pixel strategy when computing the database-query crrespondences.

from loftr.

hi,@zehongs,i have a problem when I try to reconstruction my custom data by hloc+loftr. I used round() to save the result of loftr and inserted the pixel point as keypoint. Also I inserted the matching correspondence of two images. However, I got an error in pycolmap as check failed: bundle_adjuster.Solver(&reconstruction). Can you help to solve it if you have any experience? Many thanks

from loftr.

Hi, I have not encountered this problem. Maybe you can reduce the amount of data, and manually check if the output files are all correct for COLMAP.

from loftr.