Comments (5)
Hi @Alexey322 ,
I did train from scratch for Polish language - about 14 hours dataset in total, about 9 hours of that is one speaker, other speakers' durations vary much.
I can tell you, that looking at your tensorboard and compairing to mine, I see higher loss_ctc - about 1.8 vs mine 1.3,
binarization_loss values - > 0.4, for me it was between 0.25-0.35
train/mel_loss was going toward -2.0 reaching it around 200k step, at 60k step it was around -1.7,
For val/mel_loss I had peak near 30k being -1.52 then at 200k step it was -0.75
from radtts.
Thank you for sharing the results, @szprytny . Why did you try to overfit the model and what synthesis results did you get before and after overfitting?
from radtts.
I cannot answer regarding synthesis on not overfitted model, because I used that 600k checkpoint for training second step of RADTTS++ model.
I can only say, that some of the speakers are quite biased comparing to training samples, but still for most of them you could recognize who is who :D
What is important - pronunciation is very good, there is no problem with understanding of spoken sentences, even very long ones "tongue twisters".
e.g. w gąszczu.zip
Tensorboard screenshot is from step 1 - training decoder with config_ljs_decoder.json
Then I used in 2nd step config_ljs_dap.json
to get model for synthesis.
from radtts.
Hi @szprytny, thank you for the insights! Just wondering that in your experience, what would be a sufficient amount of training steps? It's not described in the original paper, and as I am still doing initial experiments with LJSpeech, the config (https://github.com/NVIDIA/radtts/blob/main/configs/config_ljs_decoder.json) sets the total number of epochs to be 10,000,000, which seems to be way too much.
from radtts.
Hi @szprytny, thank you for the insights! Just wondering that in your experience, what would be a sufficient amount of training steps? It's not described in the original paper, and as I am still doing initial experiments with LJSpeech, the config (https://github.com/NVIDIA/radtts/blob/main/configs/config_ljs_decoder.json) sets the total number of epochs to be 10,000,000, which seems to be way too much.
That probably depends on dataset very much, but I can tell, that model is producing intelligible utterances pretty quickly for me, about 30k steps with 8 samples per batch.
I don't train model with pitch and energy conditioning anymore. I noticed that for my multispeaker data results are much worse than basic RADTTS model.
from radtts.
Related Issues (20)
- Cannot train starting from pre-trained model b/c audio files not found HOT 1
- Is it possible to do inference in real time?
- Trouble with inferencing without pitch and energy condition HOT 2
- How to slow down the speed of the response? HOT 3
- Here's a Colab notebook for using RADTTS [Documentation]
- Is it a mistake in README? HOT 1
- RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn HOT 1
- Output of voice conversion has source model's timbre, not destination models timbre HOT 2
- Certain texts in LJ speech unloadable HOT 1
- with open(config_path) as f: FileNotFoundError: [Errno 2] No such file or directory: HOT 1
- Inference: size mismatch for context_lstm.weight_ih_l0: copying a param with shape torch.Size([2080, 1044]) from checkpoint, the shape in current model is torch.Size([2080, 1040]). HOT 3
- why mix phone and word embedding HOT 1
- train decatndur HOT 2
- Inference with bgap models
- Question about spectrogram normalization HOT 1
- Training for singing models
- recommend the steps of 1st-stage training.
- Train custom voice instead of the default ljs speaker.
- Straight through on unsupervised aligner
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from radtts.