- Inverse cloze task: reference
- Random span extraction
- Indepandant croppping
- Word replacement
- Lexical: BM25
- Dense
- Learned sparse
- Fused:
-
- Scidocs
- Scifact
- TREC COVID
-
- lifestyle
- recreation
- science
- technology
- writing
- Section 1 - Introduction
- Observation: domain-specific corpus and relateness to downstream IR.
- Section 2
- Unsupervised learning in NLP: empirical observation, empirical datasets.
- IR representation learning in distant supervision: taxonomy, methods
- Bi-encoders and particular learning methods.
- Retrieval architecture design