Keypoint Scores about d2-net HOT 7 CLOSED

@PatrickRowsomeMovidius The keypoint scores of D2-Net have no absolute meaning. We only train for local uniqueness, but not global order. As such, taking the top-n / bottom-n ranked keypoints is not expected to give you the "best" keypoints.

from d2-net.

Hello @PatrickRowsomeMovidius . @tsattler is right about the scores not being trained for global ordering. Nevertheless, for HPatches sequences, I tried selecting top 2000 highest scores and the results remained similar; this doesn't guarantee that the results will be the same for all datasets, but it is worth investigating further. Thus I have some additional questions:

1. Are you using multi-scale or single-scale? Taking top-k for multi-scale is not expected to work since the feature maps are summed together for higher scales.
2. I think that keypoints = np.asarray([row for _, row in sorted(zip(scores, keypoints), key=lambda pair: pair[0])]).squeeze() sorts in ascending order so you might want to select the last N keypoints (e.g., keypoints[-N :]).

from d2-net.

I have one question, how can i get the orientation of keypoint ?

from d2-net.

We do not estimate a feature orientation (only location and eventually scale using an image pyramid). The keypoints can be considered as being upright.

from d2-net.

Hello @PatrickRowsomeMovidius . @tsattler is right about the scores not being trained for global ordering. Nevertheless, for HPatches sequences, I tried selecting top 2000 highest scores and the results remained similar; this doesn't guarantee that the results will be the same for all datasets, but it is worth investigating further. Thus I have some additional questions:

1. Are you using multi-scale or single-scale? Taking top-k for multi-scale is not expected to work since the feature maps are summed together for higher scales.

I am using the single scale option. So this should not cause an issue

2. I think that keypoints = np.asarray([row for _, row in sorted(zip(scores, keypoints), key=lambda pair: pair[0])]).squeeze() sorts in ascending order so you might want to select the last N keypoints (e.g., keypoints[-N :]).

Yes, you are right, this does sort in ascending order, this means that my results are more stable now.

Still I see quite poor performance in terms of repeatability when I attempt to only keep the best N keypoints.

from d2-net.

@mihaidusmanu Thanks for answering！

from d2-net.

@PatrickRowsomeMovidius In a recent work related to D2-Net - ASLFeat (https://arxiv.org/pdf/2003.10071.pdf), they seem to be using the soft-detection score for reducing the number of keypoints (section 5.1, paragraph Testing). I suspect this might be helpful for D2-Net features as well, but I didn't experiment with it so far.

I will close the issue. Feel free to open a new one if there are any problems.

from d2-net.