GPT2-XL and GPT-J evaluating with ZsRE about easyedit HOT 6 CLOSED

EasyEdit can be easily applied to any supported model. For example, if you want to edit GPT-J through ROME, you can run:

python run_zsre_llama2.py \
    --editing_method=ROME \
    --hparams_dir=../hparams/ROME/gpt-j-6B \
    --data_dir=./data

• The script is called run_zsre_llama2 because currently only key status on llama2 is open for MEMIT methods.
  • We will provide more status in the future and rename this script to run_zsre

Thank you for your reminder.

If it helps you solve the problem, please close this issue.

from easyedit.

Hello,

Thank you for your response. I have tried with FT, IKE and ROME and it worked for GPT2-XL and GPT-J-6B but it doesn't work for MEMIT. It is showing "ValueError: BuilderConfig '20200501.en' not found. Available: ['20220301.aa....".

I have downloaded model.layers.4.mlp.down_proj_float32_mom2_100000.npz etc and put it in the folder ./data/stats/._hugging_cache_gpt2-xl/wikipedia_stats/. What else should I do so that I can run MEMIT as well. Please let me know.

Thank you!

from easyedit.

As you can see in README, MEMIT cannot bypass the computation of second-order momentum, so it requires the npz related to Wikipedia. However, we have only opened the key status of llama2 at the moment.

• You can download the second-order momentum of gpt2-xl and gpt-j-6B used in ROME and MEMIT at website of MIT. https://rome.baulab.info/data/stats/
• If you are testing the performance on the llama2, you can simply use the npz provided by us.

from easyedit.

If it helps you solve this problem, please close the issue. And if you want to ask a new question, please open a new issue.

from easyedit.

Hello,

Thank you for your response. I have tried with FT, IKE and ROME and it worked for GPT2-XL and GPT-J-6B but it doesn't work for MEMIT. It is showing "ValueError: BuilderConfig '20200501.en' not found. Available: ['20220301.aa....".

I have downloaded model.layers.4.mlp.down_proj_float32_mom2_100000.npz etc and put it in the folder ./data/stats/._hugging_cache_gpt2-xl/wikipedia_stats/. What else should I do so that I can run MEMIT as well. Please let me know.

Thank you!

You can download the second-order momentum of gpt2-xl and gpt-j-6B used in ROME and MEMIT at website of MIT. https://rome.baulab.info/data/stats/

from easyedit.

Hello,
Thank you for your response. I have tried with FT, IKE and ROME and it worked for GPT2-XL and GPT-J-6B but it doesn't work for MEMIT. It is showing "ValueError: BuilderConfig '20200501.en' not found. Available: ['20220301.aa....".
I have downloaded model.layers.4.mlp.down_proj_float32_mom2_100000.npz etc and put it in the folder ./data/stats/._hugging_cache_gpt2-xl/wikipedia_stats/. What else should I do so that I can run MEMIT as well. Please let me know.
Thank you!

You can download the second-order momentum of gpt2-xl and gpt-j-6B used in ROME and MEMIT at website of MIT. https://rome.baulab.info/data/stats/

Thank you for your reply, downloading these weights directly is the best solution, I have updated the original reply

from easyedit.