Comments (7)
The extra model is an integral part of whisperx
and cannot fix it by deleting any files. Thanks for your shared song example, I'm trying to figure out the reason.
I am also currently working on adding support for automatically splitting long sentences in word-boundary languages, such as English. This feature will help resolve the problem of excessively long lines mentioned in issue #7.
I will be on vacation this weekend, so both of them may not be fixed promptly.
You can also try https://github.com/johnwmillr/LyricsGenius to download lyrics directly from https://genius.com/, which may provide non-timestamped lyrics with higher text quality than transcribed one. Hope your dream finds you well!
from openlrc.
I'm trying to alleviate the impact of VAD by varying the vad_onset
and vad_offset
used in whisperX, but it seems there is an existing issue on it: m-bain/whisperX#340 (the VAD parameter are fixed to be offset = onset = 0.5
). And the corresponding PR is not merged m-bain/whisperX#357.
Considering that whisperX is no longer actively maintained, I do not anticipate prompt review and merging of this pull request. As an alternative solution, I may consider switching back to faster-whisper which sacrifices some accuracy in word-level time-stamps but benefits from active maintenance.
from openlrc.
It looks like whisperx
fail to extract all of the transcriptions because of the extra VAD model:
https://github.com/m-bain/whisperX/blob/befe2b242eb59dcd7a8a122d127614d5c63d36e9/whisperx/asr.py#L259C19-L259C19
Could you please share the audio file that hasn't been transcribed accurately? Google Drive or OneDrive would be great.
from openlrc.
Thank you for your resopnse :)
I don't understand what you mean by "Extra VAD model" - does that mean there is a file i can delete to fix it? I'd gladly delete and re-download all my models, since it's automatic (?) anyway.
Anyway, I use Mega to share because they give 50G unlike google or onedrive! and I like their spirit! I uploaded the current folder here:
https://mega.nz/folder/MdVWkIxQ#jx7d9JtwRnXQEmCw-VTW2w
The song is "1_Fences.flac".
I sure hope this helps!
SOME SIDE STORY:
My dream has been AI-generated LRC files. I've spent 20 yrs making 450 or so LRC files but there are about 30,000 that I'd like to generate! Once I get the proof of concept working to the standard I want to, I plan to unleash it on 30,000 songs even if it takes months to do it all! :)
from openlrc.
The extra model is an integral part of
whisperx
and cannot fix it by deleting any files. Thanks for your shared song example, I'm trying to figure out the reason.
No problem! I'm also working with the whisper-faster.exe team and experienced similar lyrical failures and for their situation, adding the option --vad_filter False really seemed to help a lot!
I am also currently working on adding support for automatically splitting long sentences in word-boundary languages, such as English. This feature will help resolve the problem of excessively long lines mentioned in issue #7.
that sounds amazing!!!!!!
whisper-faster.exe seems to do this song fairly well with that option, but it didn't without that option:
I will be on vacation this weekend, so both of them may not be fixed promptly.
I noticed you were super fast which is why I kept talikng to you. Thnks for the warning lol! and for the hard work :)
You can also try https://github.com/johnwmillr/LyricsGenius to download lyrics directly from https://genius.com/, which may provide non-timestamped lyrics with higher text quality than transcribed one. Hope your dream finds you well!
Interesting! I use EvilLyrics ( https://www.evillabs.sk/ ) for over 20 yrs now, and MiniLyrics for 2 yrs now, and both tend to automatically download from lots of sources (and they play together, evillyrics finds stuff better and minilyrics will end up displaying it by sharing the same repository).
from openlrc.
As an update, I'm working on the same problem with whisper-faster, and it, too, was missing most lyrics, until I disabled all VAD functionality. I'm not sure if there's a way to do that with openlrc? But maybe try that, once you're back from vacation :)
from openlrc.
Update: I have released a new version that replaces whisperx
with faster-whisper
. To install this update, use
pip install openlrc -U
You can adjust the VAD parameters by using the code snippet below:
vad_options = {"threshold": 0.382}
lrcer = LRCer(vad_options=vad_options)
lrcer.run(file, prompter='base_trans', skip_trans=True)
For default parameter values, refer to:
https://github.com/zh-plus/Open-Lyrics/blob/5ec44d192f037e7ed12192d627566c1b7966c4b3/openlrc/defaults.py#L25
To understand the meaning of each parameter, visit:
https://github.com/guillaumekln/faster-whisper/blob/3b4a6aa1c22d293ddde9f08bdd31fc842086a6ea/faster_whisper/vad.py#L14
Feel free to reduce the threshold
value in order to preserve more unfiltered voices.
from openlrc.
Related Issues (5)
- Diarization HOT 1
- RuntimeError HOT 1
- AssertionError case sensitivity issue HOT 4
- exception when running HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openlrc.