Comments (6)
Hey, I quickly checked but I couldn't find any mistakes and those accuracies saved into the pickle seem correct to me. I think the confusion is with how the "all_tasks" entry is organised, which isn't very intuitive. The first list of "all_tasks" actually contains the test accuracy for the first task measured after training on each of the tasks. The n-th entry of the m-th list is the test accuracy for task m after training on task n. Sorry for that weird structure. I hope this explains it, but if not please let me know and I'll have another look.
from continual-learning.
from continual-learning.
Yes, that's correct!
from continual-learning.
from continual-learning.
Yes, that looks about right to me. The regularisation of EWC indeed hinders the network to learn new tasks (because it penalizes new parameter changes), and this effect gets larger the more tasks have been learned. You can make the network more plastic by reducing lambda (or reducing gamma might also work), which will probably increase the accuracies for the later tasks but reduce those for the earlier tasks. What the optimal values of these EWC hyper parameters are heavily depends on both the type and number of tasks. (The optimal values for different task protocols can differ by several orders of magnitude, see for example Appendix D here: https://arxiv.org/abs/1904.07734.)
from continual-learning.
Thank you again!
from continual-learning.
Related Issues (20)
- Empirical Fisher Estimation HOT 3
- Datasets more complicated than MNIST HOT 1
- Just a request
- Grad in SI HOT 4
- Wrong dataset? HOT 2
- why batch_size has to be 1 when update fisher? HOT 1
- Lower/Upper Bound Experiments HOT 2
- one little confusion about the loss_fn_kd function HOT 1
- Suspicious Precision HOT 3
- Link error HOT 2
- Reproducing BI+SI method HOT 9
- about kafc fisher infromation matrix HOT 1
- How to create Resnet34 HOT 2
- Joint training results different for different types of incremental learning? HOT 3
- Task-IL evaluation HOT 2
- Single head or multihead task incremental HOT 1
- 0 accuracy values for task-free setting HOT 9
- Whether context identity must be inferred in case of domain increment? HOT 1
- About printing results of experimental output
- Results for None ("lower target")
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from continual-learning.