imkett / pcae Goto Github PK

After Stage 1 BART VAE Finetuning, the following files were saved.
./chackpoints/yelp/BART/best_val_vae_hidden.pt

The following error occurred in Stage 2.1 PCAE Plug-in Training.
RuntimeError: Error(s) in loading state_dict for VAE: Missing key(s) in state_dict: "encoder.embed_tokens.weight", "encoder.embed_positions.weight", ...

I thought that the key name "model." was the cause, so I rewrote the code below to remove "model.".
https://github.com/ImKeTT/pcae/blob/main/train.py#L234-L238

However, I got the following error:
RuntimeError: Error(s) in loading state_dict for VAE: Missing key(s) in state_dict: "pooler.dense.weight", "pooler.dense.bias", "z_linear.weight", "linear.weight".

Please tell me how to fix it.

Thanks,

hi @ImKeTT ，evaluation.py is empty，I would like to know the calculation of evaluation metrics.

Thank you for a great job.
"Stage 1 BART VAE Finetuning" doesn't work on latest commit.
RuntimeError: mat1 and mat2 shapes cannot be multiplied (16x16 and 768x768)

Undoing the commit below seems to work.
c05f708

How can I fix it?

Hi, thanks for your great work, I'm interested in your work. Please make your code available as soon as possible.

最近我在尝试进行你们的工作的复现，但是在训练过程中我发现了一个问题。在VAE_finetuning函数中，我注意到优化器的设置为：
optimizer = AdamW(model.parameters(), lr=args.lr, correct_bias=True, no_deprecation_warning=True)
这表明优化器只优化model中，也即bart模型中的参数，实际上我发现在类VAE中，存在用于生成均值与方差的线性层：
self.z_linear = nn.Linear(self.config.hidden_size, 2 * self.nz, bias=False)
根据我的分析，这部分参数并未被上述定义的优化器优化，这是正常的吗？

Congratulations on your contribution to controlled text generation. The evaluation file evaluation.py is empty, could you provide it?