Comments (3)
Sorry, I don't understand your issue. Could you please explain it in more detail, what you want to achieve and how? Ideally show the code that leads to good or bad results.
from peft.
HI, Now, whisper can use context information to improve recognition accuracy:
And, if you want pass context information to whisper, you can use arg for cli:
https://github.com/openai/whisper/blob/main/whisper/transcribe.py#L531
parser.add_argument("--initial_prompt", type=str, default=None, help="optional text to provide as a prompt for the first window.")
when finetune the whisper model, not use "--initial_prompt", decoding result of finetuned model with using "--initial_prompt" will be worse.
from peft.
I see. I don't really have any expertise in whisper and how the initial prompt affects the outcome. But my best guess is that yes, if you want to use it, you should also use it during training, using the same logic as in the script that you linked.
from peft.
Related Issues (20)
- Add Support for IA3 Adapters in add_weighted_adapter Method, Currently facing issue that 'IA3Model' object has no attribute 'add_weighted_adapter' HOT 9
- Saved weights differ from the original model HOT 14
- DoRA uses lots of GPU VRAM due to fp32 upcasting HOT 6
- How to convert a loha safetensor trained from diffusers to webui format HOT 1
- Outdated utility function: No attribute get_module_class_from_name in FullyShardedDataParallelPlugin HOT 1
- Error while loading PEFT lora model HOT 4
- The FSDP example fails to run with "ValueError: Must flatten tensors with uniform requires_grad when use_orig_params=False" HOT 2
- OOM error while QLoRA+Deepspeed fine tuning of Llama3-70B model on 4xA100-40GB gpus HOT 2
- Support merge_and_unload for IA3 Adapters with 4-bit and 8bit Quantization models
- model merge_and_unload do not support layer_replication HOT 5
- OOM with Phi-3-mini (3.8B) on 83.5GB RAM due to LoftQ HOT 4
- Ignore keys for modules to save HOT 1
- PeftModel failing to load after finetuning. Size Mismatch Error HOT 2
- Can peft support ColumnParallelLinear? HOT 2
- eval_loss showing Nan but train_loss decreases and goes to NaN after couple of steps while fine tuning gemma model with additional vocab HOT 1
- GPU Allocation Issue (QLoRa + Llama3-8B-IT) HOT 1
- Unable to Install peft==0.7.0.dev0 HOT 2
- RuntimeError: only Tensors of floating point dtype can require gradients for QLoRA since transformers 4.40 HOT 5
- TypeError: unsupported operand type(s) for *: 'Parameter' and 'NoneType' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from peft.