Problem with r-precision about ranx HOT 8 CLOSED

Hi Oliver,

Thanks for your interest in rank_eval and the kind words.

Yesterday, when I did the last commit, I noticed something was off in the code there!
I'm going to address the problem in the next few days and come back to you.

Thanks for your feedback.

Have a good one,

Elias

from ranx.

@osf9018 the issue is now fixed.

Thanks again for your feedback!

Closing.

from ranx.

from ranx.

Sorry Oliver, I do not understand what ***@***.*** means.
Could you clarify, please?

from ranx.

from ranx.

Sorry Oliver, I am confused: I see ***@***.*** in your messages on four different browsers on two devices.

I think I probably called that measure Hits because it is the (mean) number of relevant documents retrieved for each query.
It is an integer value for each query, not a boolean one.
I am sure I saw it in some paper, but I do not recall which one at the moment.

It is a sub-function of other metrics, so I decided to expose it if someone wants to use it, but maybe I should hide it as it is not that useful anyway.

For example, precision is something like:

def precision(qrels, run, k):
    # If k is 0 use the number of retrieved documents
    k = k if k != 0 else len(run)

    return hits(qrels, run, k) / k

while recall is something like:

def recall(qrels, run, k):
    # If k is 0 use the number of retrieved documents
    # In this case k is used just to avoid useless computations
    # as we divide the number of retrieved relevant documents (hits)
    # by the number of relevant documents later
    k = k if k != 0 else len(run)

    return hits(qrels, run, k) / len(qrels)

You can use it for an analysis purpose if you want, but I suggest you stick with the other metrics for scientific evaluation / comparison.

from ranx.

from ranx.

Hi Oliver,

Thanks again for your feedback.
I will take your suggestion into consideration.

I also inform you that my tool is going to change name soon because of naming similarities with other tools.
The new name is ranx. You can already install the library with pip install ranx.

Best regards,

Elias

from ranx.