Comments (10)
Thanks for your work. With python main.py --validate --beamsearch
I now get this output:
Character error rate: 10.464338%. Word accuracy: 74.000000%.
👍
from simplehtr.
Hi,
I had to use batch normalization (BN) for a larger text recognition model, otherwise training would not converge. For this smaller model (SimpleHTR), I was able to train it without BN. BN increases the training time (it makes the model converge, and it theory it should let the model converge faster w.r.t. number of training epochs, but in practice I noticed that the computations done for BN take quite some time and therefore the training time also increased). And from a SW engineering point of view, I have to say that BN looks like a hack in TF: you plug in some node which has to be evaluated separately from the rest of the model ... pretty sure the TF team had their reasons for this, but I don't like the way it is implemented.
However, you can try to add one or two BN layers into the model and see how this influences training. I applied BN in the 3rd and 6th CNN layer. Please let me know how it performs with the SimpleHTR model if you do some experiments.
from simplehtr.
Closing because of inactivity.
from simplehtr.
With tf.layers.batch_normalization it's not so complicated, just one has to take care of train_phase.
First impression is that it helps but I have to prepare more formal tests
from simplehtr.
I'm getting over 71% word accuracy in under 20 epochs with batch normalization after every conv layer. Surprisingly, batch normalization is only about a 5% performance hit per epoch but it trains fully in about 25 epochs, so about 50% less time overall. Patch would be about 8 lines of code.
from simplehtr.
cool, if this really boosts both training time and accuracy, then we should go for it!
In case you provide a patch, could you please include the trained model (packed into model.zip)?
from simplehtr.
Ok, sounds good.
from simplehtr.
I was fooled by RMSProp being inconsistent. The training time reduction probably exists over a larger sample, but isn't as good as previously advertised (maybe 10-25%). Training ended up terminating with over 73% word accuracy so that's still a huge improvement.
from simplehtr.
Here's a visualization of batch normalization vs no batch normalization on SimpleHTR. In addition to the benefits of BN, the rate change at 10K batches is pretty noticeable. Interestingly, without BN, the model doesn't overfit to the training data.
from simplehtr.
thank you, I'll try to have a look at it on the weekend.
from simplehtr.
Related Issues (20)
- Blank line filter in dataloader doesn't quite work HOT 1
- Deep Stream HOT 1
- How to use in ML.NET c#?
- unable to build wheel for word_beam_search HOT 1
- Where can I find the tagset.txt file HOT 3
- Add feature to save train loss in summary + minor bug fix HOT 2
- Data visualization HOT 2
- pip install error: ModuleNotFoundError: No module named 'patch_ng' HOT 1
- which version of python used?
- Add cudnn64_8.dll to the Windows/System32 folder, otherwise the program cannot run properly.
- How to convert checkpoint to ONNX HOT 2
- Outdated version of tensorflow used HOT 6
- Training Model
- Training the model from scratch and error "model not found" HOT 4
- Wrong detection of words in model validation HOT 8
- Missing CITATION.cff file for repository HOT 1
- where is json HOT 9
- Hello, I'm sorry to disturb you again. How to make a front-end webpage for this project, which only needs to be able to open locally. Could you please teach me? HOT 1
- performance evaluation of the experimental results HOT 1
- AttributeError: `batch_normalization` is not available with Keras 3. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from simplehtr.