Documentation for hparams? about rome HOT 2 CLOSED

Ok, I think I understand layers – running the causal_trace.ipynb and looking at those figures suggests some candidate MLP layers that I can then provide as hparams. The mom2 ones seem to be described briefly in section E.2 of the paper, and it looks like I will have to use a code dataset for these. I think I can figure it out from here, thanks :)

from rome.

Hi @moyix! Your interpretations are correct, thanks for clarifying them -

Let us know how things go on the code task! We've never tried something like that before, so there might be some hiccups along the way + interesting new research questions. Happy to chat about them at some point, if you'd like :)

from rome.