Question about training warm-up lexical model with BM25 negatives. about led HOT 1 CLOSED

Hi, thanks for your interest!

I may not fully understand your question. "In-batch-negative" is a technique for scaling up the negative examples for more effective training. In this method, within the same batch, the positive/negative passages for other queries are ALSO used as negatives for one query, In addition to the negative examples sampled by BM25 or other sampling methods. This trick can be employed during any training phase (warm-up and continuous training) and is commonly applied in the training of other dense retrievers.

Therefore, it can integrate with the BM25 or self-mining methods, which are ways of sampling negatives.

In short, yes, it is correct.

from led.