How to only generate Precision, Recall, and f1 score when benchmarking BLINK about blink HOT 3 OPEN

Hi @clintoncheang, we did not compute f1 so you would need to implement that yourself. For other metrics, look at code here https://github.com/facebookresearch/BLINK/blob/main/blink/main_dense.py#L476 (and in similar places for cross-encoder).

from blink.

Hi @clintoncheang, we did not compute f1 so you would need to implement that yourself. For other metrics, look at code here https://github.com/facebookresearch/BLINK/blob/main/blink/main_dense.py#L476 (and in similar places for cross-encoder).

Thank you so much for the reply. I looked at the code but I am not very sure how BLINK computes the recall and accuracy. Is it based on the output from the BLINK model (Predictions for the entity, which is in string format) and compare it to the correct entities from the corresponding Test dataset (I am assuming the correct entity from the jsonl file will be the wiki_title field?) Correct me if I am wrong. Thank you so much!

from blink.

@clintoncheang: Have you looked at eval_biencoder.py script?

from blink.