It's possible to support inference of batch > 1. Could you tell me which line goes wrong?

from loftr.

import os
#os.chdir("..")
import torch
import cv2
import numpy as np
import matplotlib.cm as cm
from src.utils.plotting import make_matching_figure

from src.loftr import LoFTR, default_cfg

The default config uses dual-softmax.

The outdoor and indoor models share the same config.

You can change the default values like thr and coarse_match_type.

matcher = LoFTR(config=default_cfg)
matcher.load_state_dict(torch.load("weights/indoor_ds.ckpt")['state_dict'])
matcher = matcher.eval().cuda()

Load example images

img0_pth = "test_images/example_images/10.jpg"
img1_pth = "test_images/example_images/11.jpg"
img2_pth = "test_images/example_images/9.jpg"
img3_pth = "test_images/example_images/8.jpg"
img0_raw = cv2.imread(img0_pth, cv2.IMREAD_GRAYSCALE)
img1_raw = cv2.imread(img1_pth, cv2.IMREAD_GRAYSCALE)
img2_raw = cv2.imread(img2_pth, cv2.IMREAD_GRAYSCALE)
img3_raw = cv2.imread(img3_pth, cv2.IMREAD_GRAYSCALE)
img0_raw = np.expand_dims(cv2.resize(img0_raw, (640, 480)), axis= 0)
img1_raw = np.expand_dims(cv2.resize(img1_raw, (640, 480)), axis=0)
img2_raw = np.expand_dims(cv2.resize(img2_raw, (640, 480)), axis=0)
img3_raw = np.expand_dims(cv2.resize(img3_raw, (640, 480)), axis=0)
img0 = torch.from_numpy(np.concatenate([np.expand_dims(img0_raw, axis=0), np.expand_dims(img0_raw, axis=0), np.expand_dims(img0_raw, axis=0)])).cuda() / 255. # Modified by me to input the batch of the input
img1 = torch.from_numpy(np.concatenate([np.expand_dims(img1_raw, axis=0), np.expand_dims(img2_raw, axis=0), np.expand_dims(img3_raw, axis=0)])).cuda() / 255.
batch = {'image0': img0, 'image1': img1}

Inference with LoFTR and get prediction

with torch.no_grad():
matcher(batch)
mkpts0 = batch['mkpts0_f'].cpu().numpy()
mkpts1 = batch['mkpts1_f'].cpu().numpy()
mconf = batch['mconf'].cpu().numpy()
m_mask = batch['m_bids'].cpu().numpy()

Draw

color = cm.jet(mconf)
text = [
'LoFTR',
'Matches: {}'.format(len(mkpts0)),
]

On the base of the code of "demo_single_pair"，I used of 3x1x640x480 as the input. But I got the wrong value of batch["m_bids"]. At the same time, the dimission of "mkpts0" is Nx2. I have read the source code, and the whole process of the batch is 3. I don't know why the output batch size of "mkpts0" is 2.

from loftr.

Hi, sorry for the late reply.
The demo py file supports inference with bs>1. And no matter how many batches you use, the mkpts0_f and 'mkpts1_f are alway in the shape of Nx2. You can use m_bids to identify the parts belonging to batch0, batch1, and batchN.
In your case, to get the matched kpts0 in batch 2:

mkpts0_batch2 = batch['mkpts0_f'][batch['m_bids'] == 2].cpu().numpy()

The original purpose is to parallel the computation of fine-level.

from loftr.