question about LiLT-base about lilt HOT 4 CLOSED

Hi,
the provided LiLT-based checkpoint is trained as described in the origin paper. It should be combined with the off-the-shelf plain text pre-trained models for fine-tuning.

from lilt.

I went over the paper again, so LiLT-based initialize the text flow from the existing pre-trained English RoBERTa-BASE and when finetuning, it need to load RoBERTa-BASE again as the paper say "combine LiLTBASE with a new pre-trained RoBERTaBASE for finetuning"
If I am right, when finetuning in English, why reload RoBERTa-BASE again, it seems to be a little redundant

from lilt.

It's a nice question. As you said, the pre-trained LiLT can be directly applied in English without re-load the text-part weights, and it can get a better result. However, we reload English RoBERTa-BASE again for consistency comparison in different languages.

from lilt.

OK,I see, thanks for your reply

from lilt.