[P1] Loreft example gsm8k train gives: RuntimeError: output with shape [64, 1, 7] doesn't match the broadcast shape [64, 0, 7] about pyreft HOT 3 CLOSED

@jaymefosa thanks! yes -- it seems like you need to add a BOS token (concatenating tokenizer.bos_token_id or adding tokenizer.bos_token to the raw string) upfront and try it again.

from pyreft.

hey @jaymefosa thanks for your question!

the common reason for this error is the script does not recognize the BOS token, we need the BOS token to set the intervention location internally.

to verify this, could you print out the raw tokenized sequence for "Hello World" with your tokenizer and check? thanks!

from pyreft.

@frankaging thanks for the fast reply, the output was:

Tokenized output: {'input_ids': tensor([[9906, 4435]]), 'attention_mask': tensor([[1, 1]])}

from pyreft.