[ContinualAI Seminars]: "Multiband VAE: Latent Space Alignment for Knowledge Consolidation in Continual Learning"

[Keiland Cooper]

Hi All,

Join us this Thursday 11-10-2022, 15:30 PM UTC, for the ContinualAI Seminar, where Kamil Deja (Warsaw University of Technology) will present the paper:

Title: “Multiband VAE: Latent Space Alignment for Knowledge Consolidation in Continual Learning”
Link: https://arxiv.org/abs/2106.12196v2

Abstract: We propose a new method for unsupervised generative continual learning through realignment of Variational Autoencoder's latent space. Deep generative models suffer from catastrophic forgetting in the same way as other neural structures. Recent generative continual learning works approach this problem and try to learn from new data without forgetting previous knowledge. However, those methods usually focus on artificial scenarios where examples share almost no similarity between subsequent portions of data - an assumption not realistic in the real-life applications of continual learning. In this work, we identify this limitation and posit the goal of generative continual learning as a knowledge accumulation task. We solve it by continuously aligning latent representations of new data that we call bands in additional latent space where examples are encoded independently of their source task. In addition, we introduce a method for controlled forgetting of past data that simplifies this process. On top of the standard continual learning benchmarks, we propose a novel challenging knowledge consolidation scenario and show that the proposed approach outperforms state-of-the-art by up to twofold across all experiments and the additional real-life evaluation. To our knowledge, Multiband VAE is the first method to show forward and backward knowledge transfer in generative continual learning.


---------------
- YouTube link:
https://www.youtube.com/watch?v=fiRDqDFDplQ

- Microsoft Teams:
https://bit.ly/clai-seminars

- YouTube recordings of the previous sessions:
https://www.youtube.com/c/ContinualAI
---------------

Times:
UTC - 15:30 PM
CET - 16:30 PM
PST - 07:30 AM

Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai

Please also contact me if you want to speak at one of the next sessions!

Looking forward to seeing you all there!

All the best,
Keiland Cooper

University of California
ContinualAI Co-founder
http://kwcooper.xyz