Join us this Thursday 10-20-2022, 15:30 PM UTC, for the ContinualAI Seminar, where Rajat Saxena (University of California, Irvine) will present the paper:
See also: https://royalsocietypublishing.org/doi/full/10.1098/rstb.2019.0637Abstract
: Artificial neural networks (ANNs) tend to abruptly lose previously acquired knowledge while the information from a new item is being incorporated all at once into the network, demonstrating catastrophic forgetting. On the other hand, our brains can continually learn, fine-tune, and transfer knowledge throughout their lifespan. How does the brain achieve this? Complementary Learning Systems Theory suggests that new item information can be gradually integrated into the neocortex by interleaving novel information with existing knowledge. However, this approach is highly time-consuming and data-hungry, requiring interleaving all existing knowledge every time something new is learned. In the current study, we used deep, nonlinear ANNs to learn new items by interleaving only a subset of old items with high similarity to the new item. We chose to retrieve the previously learned class exemplars based on their similarity with the new item, as the performance on highly similar old items will be most negatively impacted by new item learning. Using such similarity-weighted interleaved learning (SWIL), we could reach performance levels comparable to that achieved by using the entire training dataset, thereby reducing the amount of data required and learning time, making SWIL a fast and data-efficient approach. In addition, we recorded from the hippocampus and visual cortex of mice, while they were exploring virtual-reality contexts with different levels of similarity. We were able to demonstrate the same similarity-weighted (SWIL) replay of sequences in the hippocampus and visual cortex in our pilot experiments. We hope that the results presented here will shed light on new item learning and memory consolidation process in the brain and ANNs.
- YouTube link: https://www.youtube.com/watch?v=kjUOhTsVl4g
- Microsoft Teams: https://bit.ly/clai-seminars
- YouTube recordings of the previous sessions: https://www.youtube.com/c/ContinualAI
Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai
Please also contact me if you want to speak at one of the next sessions!
Looking forward to seeing you all there!
All the best,
University of California