Performance of editing operations about easyedit HOT 2 CLOSED

Thank you for sharing your excellent work. I would like to know what kind of GPU configuration SERAC ROME MEND needs when editing thousands of samples respectively, and how much time will it take

Speaking empirically,

• For ROME: I do not recommend that you edit thousands of samples sequentially, as this will cause the original model weights to be completely corrupted. Its editing capacity is around 100, you can refer to the paper at https://arxiv.org/abs/2305.13172
• For SERAC: Taking the model around 6B-7B as an example, the cost of SERAC training is about 3h. Each editing takes only 0.5s, the VRAM usage is about 60GB in the training phase, and about 45GB in the inference phase
• For MEND: Its resource consumption is comparable to that of serac. However, its editing capacity is only 10 cases(see https://arxiv.org/abs/2305.13172). I also do not recommend that you use this method for sequential editing.

from easyedit.

If your problem has been resolved, please help close this issue.

from easyedit.