nianticlabs / mickey Goto Github PK

I'm a beginner with ML models for perception, so apologies if this question is basic.

In demo_inference.py there is a way to read images, which are resized to a default size of (540, 720). Then, there is a read_intrinsics function with a default parameter resize.

My question is: If input images do not have a size of (540, 720), does the resizing for the intrinsics need to be applied too? If not, when would one use it?

I am new to MVS research, so pardon me if the question is naive.
How do you ensure that the predicted depth is a correct metric depth? In other words, how do you ensure the depth predictions have correct scale. AFAIK you do not supervise your depth maps directly.

Thank you for your work. I would like to ask if the input is an image pair, one as a reference image and the other as a source image, can the source image be converted to the perspective of the reference image through the Mickey model? Similar to solving the transformation matrix between the two? Please tell me how to implement it in the code, because the demo seems to be more inclined to output the depth map and the confidence score map, but it does not give me the conversion result. Looking forward to your reply!

hello， Could you provide visuliaztion code generating Fig6 in the paper

Hi, thanks for the interesting work. We'd like to use this in the context of detecting loop closures for visual SLAM.

As the dataset we are using does not really line up well with what MicKey was trained on (indoor scenes of partially constructed buildings), I don't expect MicKey to perform well out of the box. However, I'd like to somehow quantify "how bad is the domain gap".

In the website, you show a confidence metric between two images:

Is it possible to obtain this number from the pose matcher? The output of the model from demo_inference.py is:

Data keys: dict_keys(['image0', 'image1', 'K_color0', 'K_color1', 'kps0_shape', 
'kps1_shape', 'depth0_map', 'depth1_map', 'down_factor', 'kps0', 'depth_kp0', 
'scr0', 'kps1', 'depth_kp1', 'scr1', 'scores', 'dsc0', 'dsc1', 'kp_scores', 'final_scores', 
'R', 't', 'inliers', 'inliers_list'])

Is the confidence in one of these output variables?

(a more general question: What do all of these mean?)

Thanks for your work!
I want to know if this work can do cross-domain image matching? For example, visible light images and infrared images?