Git Product home page Git Product logo

continual_learning_papers's Introduction

Continual Learning Literature

This repository is maintained by Massimo Caccia and Timothée Lesort don't hesitate to send us an email to collaborate or fix some entries ({massimo.p.caccia , t.lesort} at gmail.com). The automation script of this repo is adapted from Automatic_Awesome_Bibliography.

For contributing to the repository please follow the process here

Outline

Classics

Argues knowledge transfer is essential if robots are to learn control with moderate learning times

Introduces CL and reveals the catastrophic forgetting problem

Surveys

introduces a super simple methods that outperforms almost all methods in all of the CL benchmarks. We need new better benchamrks

Extensive empirical study of CL methods (in the multi-head setting)

An extensive review of CL

An extensive review of CL methods in three different scenarios (task-, domain-, and class-incremental learning)

Influentials

More efficient GEM; Introduces online continual learning

Proposes desideratas and reexamines the evaluation protocol

Proposes a reference architecture for a continual learning system

A model that alliviates CF via constrained optimization

Introduces generative replay

Investigates CF in neural networks

New Settings or Metrics

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted

(title is a good enough summary)

Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning

method for compositional continual learning of sequence-to-sequence models

Regularization Methods

continual learning for non-stationary data using Bayesian neural networks and memory-based online variational Bayes

Improved results and interpretation of VCL.

Introduces VCL with uncertainty measured for neurons instead of weights.

functional regularisation for Continual Learning: avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function

Introduces an optimizer for CL that relies on closed form updates of mu and sigma of BNN; introduce label trick for class learning (single-head)

Conceptor-Aided Backprop (CAB): gradients are shielded by conceptors against degradation of previously learned tasks

Formalizes the shortcomings of multi-head evaluation, as well as the importance of replay in single-head setup. Presenting an improved version of EWC.

A new P\&C architecture; online EWC for keeping the knowledge about the previous task, knowledge for keeping the knowledge about the current task (Multi-head setting, RL)

Improves on VCL

Importance of parameter measured based on their contribution to change in the learned prediction function

Synaptic Intelligence (SI). Importance of parameter measured based on their contribution to change in the loss.

Distillation Methods

Introducing global distillation loss and balanced finetuning; leveraging unlabeled data in the open world setting (Single-head setting)

Introducing bias parameters to the last fully connected layer to resolve the data imbalance issue (Single-head setting)

Introducing an expert of the current task in the knowledge distillation method (Multi-head setting)

Finetuning the last fully connected layer with a balanced dataset to resolve the data imbalance issue (Single-head setting)

Functional regularization through distillation (keeping the output of the updated network on the new data close to the output of the old network on the new data)

Binary cross-entropy loss for representation learning & exemplar memory (or coreset) for replay (Single-head setting)

Rehearsal Methods

More efficient GEM; Introduces online continual learning

projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task

sample selection as a constraint reduction problem based on the constrained optimization view of continual learning

Controlled sampling of memories for replay to automatically rehearse on tasks currently undergoing the most forgetting

Uses stacks of VQ-VAE modules to progressively compress the data stream, enabling better rehearsal

A model that alliviates CF via constrained optimization

Binary cross-entropy loss for representation learning & exemplar memory (or coreset) for replay (Single-head setting)

Generative Replay Methods

introdudes Dynamic generative memory (DGM) which relies on conditional generative adversarial networks with learnable connection plasticity realized with neural masking

Extensive evaluation of CL methods for generative modeling

Extensive evaluation of generative replay methods

smarter Generative Replay

Introduces generative replay

Dynamic Architectures or Routing Methods

Proposes a random path selection algorithm, called RPSnet, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse

Proposes a reference architecture for a continual learning system
  • Progressive Neural Networks , (2016) by {Rusu}, A.~A., {Rabinowitz}, N.~C., {Desjardins}, G., {Soyer}, H., {Kirkpatrick}, J., {Kavukcuoglu}, K., {Pascanu}, R. and {Hadsell}, R. [bib]
Each task have a specific model connected to the previous ones

Hybrid Methods

Learning task-conditioned hypernetworks for continual learning as well as task embeddings; hypernetwors offers good model compression.

Approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. All enforced in an iterative manner

Continual Few-Shot Learning

proposes a new continual few-shot setting where spacial and temporal context can be leveraged to and unseen classes need to be predicted

(title is a good enough summary)

Proposes a new approach to CL evaluation more aligned with real-life applications, bringing CL closer to Online Learning and Open-World learning

defines Online Meta-learning; propsoses Follow the Meta Leader (FTML) (~ Online MAML)

Meta-learns a tasks structure; continual adaptation via non-parametric prior

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

Introduces What & How framework; enables Task Agnostic CL with meta learned task inference

Meta-Continual Learning

Proposes an online replay-based meta-continual learning algorithm with learning-rate modulation to mitigate catastrophic forgetting

Follow-up of OML. Meta-learns an activation-gating function instead.

Introduces Learns how to continually learn (OML) i.e. learns how to do online updates without forgetting.

Learning MAML in a Meta continual learning way slows down forgetting

Lifelong Reinforcement Learning

Formulates an online learning procedure that uses SGD to update model parameters, and an EM with a Chinese restaurant process prior to develop and maintain a mixture of models to handle non-stationary task distribution

Continual Generative Modeling

Introduces unsupervised continual learning (no task label and no task boundaries)

Extensive evaluation of CL methods for generative modeling

Applications

a healthcare-specific replay-based method to mitigate destructive interference during continual learning

method for compositional continual learning of sequence-to-sequence models

HTM applied to real-world anomaly detection problem

HTM applied to a prediction problem of taxi passenger demand

Thesis

Workshops

continual_learning_papers's People

Contributors

eugenium avatar kibok90 avatar optimass avatar piyush-555 avatar tdiethe avatar tlesort avatar yudahit avatar

Stargazers

avatar

Forkers

yuanwanglll

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.