Pre-trained models for uniCOIL about coil HOT 6 CLOSED

@nirmal2k the model mentioned above is a uniCOIL model
castorini/unicoil-d2q-msmarco-passage
We dont have cls_proj for unicoil.
In the huggingface model, the tok_proj parameters are under key coil_encoder.tok_proj. inside pytorch_model.bin
Here we defined the class to directly load the model from huggingface
https://github.com/castorini/pyserini/blob/88479fdecdb5d44934f4d91be56487148ec6601c/pyserini/encode/_unicoil.py#L27

from coil.

Yes, we have our checkpoint released on huggingface
castorini/unicoil-d2q-msmarco-passage

And here is a reproduce doc for your reference (in pyserini):
https://github.com/castorini/pyserini/blob/master/docs/experiments-unicoil.md

from coil.

Great, thank you!

My goal is to use uniCOIL and COIL with a different corpus. Is there also any guide that covers how to apply it to a different text collection (e.g., how to prepare a corpus input data that can be used with the run_marco.py script for text encoding)?

My current understanding is that I would need to train a model (or use the model you just shared). Then I need to apply the model to pre-process the text collection using the commands described in the Encoding subsection, then convert the run_marco.py's output to an Anserini's data JSONL collection input format using doc_emb_to_jsonl.py, and the follow the standard Anserini commands (with the necessary adjustments) described in the doc that you linked.

Is this correct?

from coil.

Yes it is correct.

The current input (the json) file is tokenized passage in following format: "text": " [SEP] ".
In uniCOIL work, we generate expansions by docTTTTTquery.
For your own corpus, you can either use your own expansion for passage or simply do it without expansion.

from coil.

Thanks very much!

from coil.

I think I'm missing something. May I know where to find tok_proj and cls_proj models? Is it a part of the model at huggingface?

from coil.