How to apply this to models without tie_embedding_weights? about mucoco HOT 1 CLOSED

Hi,

You are right, if the embeddings are not tied, we would have to use the output embedding table in which case the language model will not provide gradients to the input being generated. So at any update, a token gets signals either from the past tokens (via the usual forward pass) or from the future (via the gradients). If we use the output embeddings, the tokens won't receive gradients from the future tokens.

While it will affect the performance a little bit, the approach can still generate good samples with control because the token will still get gradients on the entire sequence from the control functions, which can now be defined as functions of output embeddings. I did some small-scale experiments with this setting and didn't observe a significant drop in performance (for toxicity avoidance). In general, I have observed that the gradients from the future tokens are not very meaningful especially if the sequence length is long.

Hope this answers your question.

Thanks,

from mucoco.