microsoft / coco-lm Goto Github PK

Hi,
Thanks for the great work!
Do u plan to provide the pretrain code and script?

Hi,

Thanks for your great work!

I want to further train my language model with COCO-LM objectives. I didn't find the code for the further pre-training. Will you provide the code?

Code Ref

The use of THC is not compatible with torch>1.1.

Is there any plan to support torch>1.1?

How can I test on the glue dataset with huggingface

Hi @yumeng5 ,

Could you please share the scripts to convert the fairseq base models to the Huggingface compatible models?

Thanks,
Kamal

We want to use coco-LM to do post-ptrtain（like MLM）， but the main transformer �didn‘t seen MLM task，we don't know if it‘s feasible

having this issue in GLUE preprocessing, may be due to issues in pretrained sentencepiece or any other problem?