Comments (3)
Hi, thanks for your question. I expect that this is the final performance on a class-incremental learning experiment? (So using the flag --scenario=class
.) In that case, if for example you use no specific continual learning methods, the network tends to overfit on the classes in the last task. In the last task, the network only sees the classes from that task, so then it learns to never predicted classes from the previous tasks anymore.
from continual-learning.
Hi, first of first, nice repo. thank you for providing this.
I have the same confusion. I tried the ewc method by:
./main.py --ewc --lambda=5000 --visdom
and by default, I think I use --experiment=splitMNIST
and --scenario=class
. The final result looks like:
` Precision on test-set:
- Task 1: 0.0000
- Task 2: 0.0000
- Task 3: 0.0000
- Task 4: 0.0000
- Task 5: 0.9939
=> Average precision over all 5 tasks: 0.1988`
Based on the comment, I guess I should tune the learning rate to make it less overfitting? How about the lambda parameter? Can you provide a reasonable parameters in comment so I could try to make sense on the result?
from continual-learning.
Hi @gentlegy, apologies for the late reply. In the class-incremental learning scenario, as pointed out in the paper accompanying this code repository (https://arxiv.org/abs/1904.07734), the method EWC indeed does not work well and the results that you found are typical for EWC in this scenario. Methods using some form of replay (e.g., DGR, RtF, ER, A-GEM or iCaRL) typically perform much better with class-incremental learning.
from continual-learning.
Related Issues (20)
- Empirical Fisher Estimation HOT 3
- Datasets more complicated than MNIST HOT 1
- Just a request
- Grad in SI HOT 4
- Wrong dataset? HOT 2
- why batch_size has to be 1 when update fisher? HOT 1
- Lower/Upper Bound Experiments HOT 2
- one little confusion about the loss_fn_kd function HOT 1
- Link error HOT 2
- Reproducing BI+SI method HOT 9
- about kafc fisher infromation matrix HOT 1
- How to create Resnet34 HOT 2
- Joint training results different for different types of incremental learning? HOT 3
- Task-IL evaluation HOT 2
- Single head or multihead task incremental HOT 1
- 0 accuracy values for task-free setting HOT 9
- Whether context identity must be inferred in case of domain increment? HOT 1
- About printing results of experimental output
- Results for None ("lower target")
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from continual-learning.