Comments (5)
I think it sounds like a good idea if you start out with the gpt2 script and then I can do the t5 script. I think that would be a nice way to do it and I can understand much more by following along with your work.
Btw I'm on the huggingface discord if you want to messag me my username is birger
from peft.
@pacman100 @younesbelkada I could work on this if there is any interest. What would be the first steps?
from peft.
Thanks for your interest @BirgerMoell !! That would be definitely helpful
We need to check first the compatbility of peft
and trl
(i.e. it might not directly work out of the box but we will need to double check), so we will most likely adapt the gpt2 script ourselves just to make sure we fix things that needs to be fixed, then you can help us maybe converting the t5 script? Let me know what do you think!
If you feel you want to take the challenge and start converting the gpt2 script feel free to do it and we'll be happy to review your Pull Request / code changes ! 🔥
from peft.
The PR huggingface/trl#163 should add peft
support on trl
, we may need to convert the other scripts such as t5 summarization etc as well once this PR gets merged!
from peft.
Hi everyone,
Closing this issue as we now officially support PEFT in TRL , making it possible to apply RLHF + PEFT
from peft.
Related Issues (20)
- Inference with different LoRA adapters in the same batch does not use the correct module_to_save classifier HOT 4
- Deprecation: Transformers will no longer support `past_key_values` to be tuples HOT 6
- Training with Multiple LoRAs HOT 4
- Changing lora_alpha during inference HOT 5
- Documentation and optimization of X-LoRA HOT 3
- MobileViT does not work with Inference with different LoRA adapters in the same batch HOT 3
- hyper-paramter `r` in adalora_confg and lora_config HOT 1
- PEFT QLoRA with DeepSpeed ZeRO3 on multiple GPUs not working as in documentation HOT 6
- AttributeError: 'NoneType' object has no attribute 'shape' HOT 14
- Merging adapters for BF16 model on CPU is very slow HOT 1
- LoraConfig.__init__() got an unexpected keyword argument 'layer_replication' when using custom model handler HOT 2
- RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn HOT 1
- LoftQ init without scaling HOT 1
- Gumbel-sigmoid (poly) HOT 2
- DoRA with fsdp model cannot work HOT 14
- TypeError: WhisperForConditionalGeneration.forward() got an unexpected keyword argument 'input_ids' HOT 4
- Linear4bit not supported error when running QDoRA_finetuning.ipynb cookbook HOT 1
- Support optimum-quanto
- Lora initialisation with olora and pissa not working with quantisation. HOT 7
- Issue with PEFT model save_pretrained HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from peft.