[ContinualAI Seminars]: "NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks"

[Keiland Cooper]

Hi All,

Join us this Thursday 11-03-2022, 15:30 PMUTC, for the ContinualAI Seminar, where Mustafa Burak Gurbuz (Georgia Institute of Technology) will present the paper:

Title: “NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks”
Link: https://arxiv.org/abs/2206.09117

Abstract: The goal of continual learning (CL) is to learn different tasks over time. The main desiderata associated with CL are to maintain performance on older tasks, leverage the latter to improve learning of future tasks, and to introduce minimal overhead in the training process (for instance, to not require a growing model or retraining). We propose the Neuro-Inspired Stability-Plasticity Adaptation (NISPA) architecture that addresses these desiderata through a sparse neural network with fixed density. NISPA forms stable paths to preserve learned knowledge from older tasks. Also, NISPA uses connection rewiring to create new plastic paths that reuse existing knowledge on novel tasks. Our extensive evaluation on EMNIST, FashionMNIST, CIFAR10, and CIFAR100 datasets shows that NISPA significantly outperforms representative state-of-the-art continual learning baselines, and it uses up to ten times fewer learnable parameters compared to baselines. We also make the case that sparsity is an essential ingredient for continual learning.


---------------
- YouTube link:
https://www.youtube.com/watch?v=VOcst1NuDK4

- Microsoft Teams:
https://bit.ly/clai-seminars

- YouTube recordings of the previous sessions:
https://www.youtube.com/c/ContinualAI
---------------

Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai

Please also contact me if you want to speak at one of the next sessions!

Looking forward to seeing you all there!

All the best,
Keiland Cooper

University of California
ContinualAI Co-founder
http://kwcooper.xyz