chrisdonahue / ilm Goto Github PK

Is it easy to use the framework to adapt to another pre-trained model? For example, BART?

I'd like to be able to do a version of sentence infilling that allows for conditioning the generation on a leading token—i.e., semantically prompting the generated infill sentence. In your estimation, would it be a big job to enable this kind of generation? I'm thinking of the way that initial words like "however", "therefore", "further", and so on can have a strong semantic effect on the kind of sentence infill generated.

I have set up the dataset according to your instructions. However, when I tried to train using your code, I encountered the following error. I have spent some time debugging it without a solution. Can you please advise on how to resolve this?

Where did you implement the early stopping based on PPL on the validation set in the training script? Thanks

Hi,

Thanks for releasing this! I've only just started to play with it but managed to get the example from the Jupyter Notebook working without any problems.

My questions:

1. Is it possible to generate a phrase of a variable number of tokens (e.g. <= 2) in a mask?
   E.g. I like to _ on Tuesdays.
   sing, eat cake, eat bread, dance, ...
   I tried using the <|startofinfill|> and <|endofinfill|> tokens but the output didn't make sense:

I am looking forward to<|startofinfill|><|endofinfill|> in this year summer camps. 
--------------------------------------------------------------------------------
I am looking forward to We have my entire gym membership. My mom's husband has taken my sister and I to their house. I can not wait to go. in this year summer camps.

2. Can we also get a top k list of candidates and probabilities for each mask?

Thanks!

It continually says I am missing certain things. I am new, and would really like to try it. Could I please have some guidance?

I have a custom tokenizer that's just a BertTokenizer with a custom vocab, which was used when pretraining my GPT-2 model. I'm trying to specify it to the train_ilm.py script, but I hit a NotImplemented error that I'm not sure how to solve. Any thoughts?

when i git clone, the compute talked me [email protected]: Permission denied (publickey). can you help me ?

I have a fine-tuned GPT-2 model, trained on a specific text domain, on english language. The model input has been tokenized with SentencePiece. How to adapt that model to ILM if possible?

Thank you.