[BUG] `MRR@1` is not equal `Recall@1` about ranx HOT 3 CLOSED

MRR@k is only equal to Recall@k if you have exactly one positive per query. Unfortunately, there are many meanings for Recall, but at least for trec eval (which ranx follows) it always uses positives_found/total_positives, even if k is smaller than the amount of positives you have.

For the other definitions, one I have seen called as R_cap (using positives_found/min(k,total_positives)) or Success ( the function "found any positives @ k").

from ranx.

Recall@1 is equal to MRR@1 only if you have exactly 1 relevant document per query, otherwise they are not as Recall takes into consideration the total number of relevant documents (retrieved + non-retrieved).

You can find the definition of both metrics in ranx's documentation (Recall - MRR) and Wikipedia (Recall - MRR).

from ranx.

I see,
Thank you for your answers.

from ranx.