Comments (1)
Problem 1: As you can see in README, IKE
is not currently supported in the example
module. If you want to use IKE to edit the model, you need to to first encode train_set samples and take train_ds
as an argument to edit.
like: https://github.com/zjunlp/EasyEdit/blob/main/edit.py#L417
Problem 2: In example
module (https://github.com/zjunlp/EasyEdit/tree/main/examples), we provide the script to reprodue experiment. The final edited metric is stored in results.json
as follows, and you only need to calculate the corresponding average to get the final edited performance. Later on, we will provide summarize.py
to summarize the metrics.
{
"post": {
"rewrite_acc": ,
"rephrase_acc": ,
"locality": {
"YOUR_LOCALITY_KEY": ,
//...
},
"portablility": {
"YOUR_PORTABILITY_KEY": ,
//...
},
},
"pre": {
"rewrite_acc": ,
"rephrase_acc": ,
"portablility": {
"YOUR_PORTABILITY_KEY": ,
//...
},
}
}
Problem 3: You can solve the OOM problem in the following three ways
- git clone code, using a GPU with larger memory (A800, 80GB was used in the original paper)
- If your resources are limited, consider using
device_map='auto'
for model parallel(multiple gpus needed), you can refer to the official description of huggingface here- the location of the code you changed should be in (https://github.com/zjunlp/EasyEdit/blob/main/easyeditor/editors/editor.py#L71)
- Reduce the value of k
from easyedit.
Related Issues (20)
- MEND编辑方法中未知的参数archive HOT 3
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! HOT 2
- out of gpu memory HOT 1
- Implement performance on ground truth in pre and post measures HOT 1
- Git too fast, can't see the result. HOT 1
- 能否出个qwen的相关tutorial HOT 4
- About Baichuan HOT 2
- I use two A40 GPUs and KN to edit llama2-7b. But the result is still OOM. HOT 5
- qwen14b HOT 2
- Multimodal model editing HOT 5
- MMEdit support LLaVA HOT 1
- GPU memory requirement for each method HOT 1
- 为何使用条件生成? HOT 2
- Failed to reproduce your results for ROME HOT 2
- How can I rebuild the results in the Paper "Editing Large Language Models: Problems, Methods, and Opportunities" HOT 2
- Multimodal Large Language Model Editing HOT 2
- Edit Multimodal Large Language Models HOT 7
- 综述论文当中sequential editing的复现问题 HOT 10
- Question about the evaluation metric HOT 3
- OOM issue with "edit_IKE_MiniGPT4_VQA". HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from easyedit.