ContinualAI Reading Group: "Efficient Continual Learning with Modular Networks and Task-Driven Priors"

14 views
Skip to first unread message

Vincenzo Lomonaco

unread,
Jan 11, 2021, 8:17:24 AM1/11/21
to Continual Learning & AI News
Dear All,

This Friday 15-1-2021, 5.30pm CET, for the ContinualAI Reading GroupTom Veniat (Sorbonne Universite) will present the paper:

TitleEfficient Continual Learning with Modular Networks and Task-Driven Priors

AbstractExisting literature in Continual Learning (CL) has focused on overcoming catastrophic forgetting, the inability of the learner to recall how to perform tasks observed in the past. There are however other desirable properties of a CL system, such as the ability to transfer knowledge from previous tasks and to scale memory and compute sub-linearly with the number of tasks. Since most current benchmarks focus only on forgetting using short streams of tasks, we first propose a new suite of benchmarks to probe CL algorithms across these new axes. Finally, we introduce a new modular architecture, whose modules represent atomic skills that can be composed to perform a certain task. Learning a task reduces to figuring out which past modules to re-use, and which new modules to instantiate to solve the current task. Our learning algorithm leverages a task-driven prior over the exponential search space of all possible ways to combine modules, enabling efficient learning on long streams of tasks. Our experiments show that this modular architecture and learning algorithm perform competitively on widely used CL benchmarks while yielding superior performance on the more challenging benchmarks we introduce in this work.

The event will be moderated by: Vincenzo Lomonaco.  

---------------
- Eventbrite event (to save it in you calendar and get reminders)https://www.eventbrite.it/e/efficient-continual-learning-with-modular-networks-and-task-driven-priors-tickets-136274599997
- Microsoft Teamsclick here to join
- YouTube recordings of the previous sessionshttps://www.youtube.com/c/ContinualAI
---------------

Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai

Best regards,

ContinualAI President
Vincenzo Lomonaco
Reply all
Reply to author
Forward
0 new messages