trec_dl's Introduction

Deep Learning Track - TREC 2019

Attempt on deep learning track challenge in TREC 2019 - https://microsoft.github.io/TREC-2019-Deep-Learning/

Tasks

Passage Ranking
Document Ranking

Baselines

For Passage Ranking

Official Duet baseline 👉 check here

Bert Baseline 👉 here

Updated Baseline here based on Duetv2 👉 here

Reproducing Bert Baseline : In Progress...
For Document Ranking

In Progress...

trec_dl's People

Contributors

Stargazers

Watchers

trec_dl's Issues

Document Ranking

Passage Ranking

Passage Ranking Leaderboard here: http://www.msmarco.org/leaders.aspx [submission after March 2019]

Full ranking (based on BERT baselines): https://github.com/nyu-dl/dl4ir-doc2query

### Update

Passage Level Train and Inference (BERT-MaxP, BERT-SumP) ➡️ https://colab.research.google.com/drive/1YAj_yA7R8Sv9QaJkKfjC0sA0vpeEh3dC
👇
The output of the above will be a file of scores for each document/passage. We can align with the document/passage ids in the processed .trec.with_json file

The input to the BERT-reranker is a list of .trec.with_json files. Each line is in the form of:

10 Q0 trec12-en0008-49-09144_passage-0 1 -5.66498569 passage # {"doc": {"title": "Personal Keyboards reviews - Keyboard-Reviews.com", "body": "personal keyboards reviews...}}

Recommend Projects