Comments (9)
Not really sure, but the learning rate seems to be too high.
In this work we used a learning rate of 0.001
from avsr-tf1.
Hi @javadpeyman @georgesterpu,
I have solved the problem, for me the reasons for that can be:
- video_processing set to None, after I change that, the results also variant.
- maybe you have wrong audio or video features.
I run now learning rate with 0.001 and it works now
@georgesterpu thank you for your contribution and help.
from avsr-tf1.
@georgesterpu , sry, have this problem solved.
from avsr-tf1.
Hi @wentaoxandry
Apologies for replying so late.
Would you mind explaining how you were able to solve this problem ?
Other people may have the same question, and they may find your answer useful.
from avsr-tf1.
Hi @georgesterpu,
I made a mistake, I have changed video_processing to None, Then I changed that to feature, it is ok.
At the beginning, the results changed, here is the results from the first 5 epochs
Test/6353179190214044575/00010 THET THANG [IT'S ALL BASED ON THIS 58 PAGE REPORT PREPARED FOR THE EUROPEAN COMMISSION]
Test/6379224730890468911/00025 THENG [GET DOWN TO YOUR LOCAL ONE]
Test/6338130913429528164/00028 THENG [VISITING ALMOST ALL PARTS OF THE EMPIRE]
Test/6362096830810797583/00021 THENG [IT'S A TERRIBLE IDEA]
Test/6350222105230808621/00007 THENG [I'M WITH MARK TWAIN LIES]
Test/6359580409472005362/00001 THENG [THEY HAVE NO IDEA WHAT TO EXPECT]
Test/6384829663342218937/00039 THENG [REGARDLESS OF WHAT HAPPENS]
Test/6335676339489461610/00014 THENG [WE DON'T WANT TO SPEND MORE THAN]
Test/6330311066473698535/00011 THENG [AND FOR ME THE SURPRISE WAS]
Test/6382618614178237455/00017 THET THANG [TO DIVIDE THE ROOM INTO TWO PARTS]
Test/6350921755403330389/00056 THENG [GIVING HOPE AND CARE TO OTHERS IS SOMETHING OUR ARMED FORCES DO WITHOUT QUESTION]
But now, after 20 epochs it all the same again:
Test/6353179190214044575/00010 THING [IT'S ALL BASED ON THIS 58 PAGE REPORT PREPARED FOR THE EUROPEAN COMMISSION]
Test/6379224730890468911/00025 THING [GET DOWN TO YOUR LOCAL ONE]
Test/6338130913429528164/00028 THING [VISITING ALMOST ALL PARTS OF THE EMPIRE]
Test/6362096830810797583/00021 THING [IT'S A TERRIBLE IDEA]
Test/6350222105230808621/00007 THING [I'M WITH MARK TWAIN LIES]
Test/6359580409472005362/00001 THING [THEY HAVE NO IDEA WHAT TO EXPECT]
Test/6384829663342218937/00039 THING [REGARDLESS OF WHAT HAPPENS]
Test/6335676339489461610/00014 THING [WE DON'T WANT TO SPEND MORE THAN]
Test/6330311066473698535/00011 THING [AND FOR ME THE SURPRISE WAS]
Test/6382618614178237455/00017 THING [TO DIVIDE THE ROOM INTO TWO PARTS]
Test/6350921755403330389/00056 THING [GIVING HOPE AND CARE TO OTHERS IS SOMETHING OUR ARMED FORCES DO WITHOUT QUESTION]
Here are some logs:
Average batch_loss as epoch 1 is 3.4890882644100465
--- new best score ---
Average batch_loss as epoch 2 is 4.203346429478308
Average batch_loss as epoch 3 is 3.5442280127395014
Average batch_loss as epoch 4 is 2.9710363551943444
--- new best score ---
Average batch_loss as epoch 5 is 2.7390092682147373
--- new best score ---
character: 86.2528% word: 100.0000%
Average batch_loss as epoch 6 is 2.6276948267381877
--- new best score ---
Average batch_loss as epoch 7 is 2.629956953446564
Average batch_loss as epoch 8 is 2.664978136976807
Average batch_loss as epoch 9 is 2.7064000020609633
Average batch_loss as epoch 10 is 2.6471189719302806
character: 77.7236% word: 95.1781%
Average batch_loss as epoch 11 is 2.6187857994875308
--- new best score ---
Average batch_loss as epoch 12 is 2.631853339662216
Average batch_loss as epoch 13 is 2.65958504972991
Average batch_loss as epoch 14 is 2.6713306309767146
Average batch_loss as epoch 15 is 2.828360452390359
character: 79.9721% word: 99.0378%
Average batch_loss as epoch 16 is 2.6376371674409316
reduce learning rate to: 0.085
Average batch_loss as epoch 17 is 2.6348431855748653
--- new best score ---
Average batch_loss as epoch 18 is 3.4625331254972926
Average batch_loss as epoch 19 is 3.207513002875429
Average batch_loss as epoch 20 is 2.879233657844812
character: 88.1714% word: 99.9441%
Average batch_loss as epoch 21 is 3.2664754792276622
Average batch_loss as epoch 22 is 3.2363471412757416
reduce learning rate to: 0.07225000000000001
Average batch_loss as epoch 23 is 2.963414747152269
--- new best score ---
I used learning rate as 0.1, and if the results not getting better in 5 epochs, the learning rate will reduce 25%. I want to ask if I should just wait until better loss error, then the results will be fine? Or I make some mistake again?
Thank you so much
from avsr-tf1.
same problem...
@wentaoxandry did you solve the problem?
@georgesterpu any idea?
from avsr-tf1.
@wentaoxandry thank you,
did you run with audio? I want to just use video, without audio?
from avsr-tf1.
@javadpeyman I run with both audio and video features. Maybe there is something wrong with your video features?
from avsr-tf1.
@wentaoxandry thank you so much. I will check that.
from avsr-tf1.
Related Issues (20)
- Inquiry about some parameter selection reason HOT 5
- awgn: out of bounds when sampling noise clip HOT 1
- [AMSGrad] get running error in audio only training HOT 3
- How do I change loss function to CTC?? HOT 2
- [feature] minimum data length for stack log mel feature HOT 2
- visemes and phonemes mapping HOT 1
- How many epochs should i train? HOT 2
- How can i change this av_align model for applying to audio2video?
- What should I do to reproduce the results of the paper? HOT 5
- How can I solve this problem HOT 6
- Inquiry about aus csv generation HOT 2
- What do the folders speaker-dependence and speaker-independence stand for? HOT 2
- How to run this program on multiple GPUs HOT 5
- The tfrecord files HOT 8
- use video-only on LRS2 HOT 1
- The result in noisy environment HOT 6
- KeyError: 'aus' when running run_audiovisual.py HOT 2
- run_audiovisual.py HOT 1
- Ask about epochs and learning rate HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from avsr-tf1.