[ContinualAI Seminars]: "Rapid learning of new items using similarity-weighted interleaved learning (SWIL)"
6 views
Skip to first unread message
Keiland Cooper
Oct 17, 2022, 4:27:59 PM10/17/22
to Continual Learning & AI News
Hi All,
Join us this Thursday 10-20-2022, 15:30 PM UTC, for the ContinualAI Seminar, where Rajat Saxena (University of California, Irvine) will present the paper:
Title: “Rapid learning of new items using similarity-weighted interleaved learning (SWIL)”
Link: https://www.pnas.org/doi/10.1073/pnas.2115229119
Join us this Thursday 10-20-2022, 15:30 PM UTC, for the ContinualAI Seminar, where Rajat Saxena (University of California, Irvine) will present the paper:
Title: “Rapid learning of new items using similarity-weighted interleaved learning (SWIL)”
Link: https://www.pnas.org/doi/10.1073/pnas.2115229119
See also: https://royalsocietypublishing.org/doi/full/10.1098/rstb.2019.0637
Abstract: Artificial neural networks (ANNs) tend to abruptly lose previously acquired knowledge while the information from a new item is being incorporated all at once into the network, demonstrating catastrophic forgetting. On the other hand, our brains can continually learn, fine-tune, and transfer knowledge throughout their lifespan. How does the brain achieve this? Complementary Learning Systems Theory suggests that new item information can be gradually integrated into the neocortex by interleaving novel information with existing knowledge. However, this approach is highly time-consuming and data-hungry, requiring interleaving all existing knowledge every time something new is learned. In the current study, we used deep, nonlinear ANNs to learn new items by interleaving only a subset of old items with high similarity to the new item. We chose to retrieve the previously learned class exemplars based on their similarity with the new item, as the performance on highly similar old items will be most negatively impacted by new item learning. Using such similarity-weighted interleaved learning (SWIL), we could reach performance levels comparable to that achieved by using the entire training dataset, thereby reducing the amount of data required and learning time, making SWIL a fast and data-efficient approach. In addition, we recorded from the hippocampus and visual cortex of mice, while they were exploring virtual-reality contexts with different levels of similarity. We were able to demonstrate the same similarity-weighted (SWIL) replay of sequences in the hippocampus and visual cortex in our pilot experiments. We hope that the results presented here will shed light on new item learning and memory consolidation process in the brain and ANNs.
---------------
- YouTube link:
https://www.youtube.com/watch?v=kjUOhTsVl4g
- Microsoft Teams:
https://bit.ly/clai-seminars
- YouTube recordings of the previous sessions:
https://www.youtube.com/c/ContinualAI
---------------
Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai
Please also contact me if you want to speak at one of the next sessions!
Looking forward to seeing you all there!
All the best,
Keiland Cooper
University of California
ContinualAI Co-founder
http://kwcooper.xyz
Abstract: Artificial neural networks (ANNs) tend to abruptly lose previously acquired knowledge while the information from a new item is being incorporated all at once into the network, demonstrating catastrophic forgetting. On the other hand, our brains can continually learn, fine-tune, and transfer knowledge throughout their lifespan. How does the brain achieve this? Complementary Learning Systems Theory suggests that new item information can be gradually integrated into the neocortex by interleaving novel information with existing knowledge. However, this approach is highly time-consuming and data-hungry, requiring interleaving all existing knowledge every time something new is learned. In the current study, we used deep, nonlinear ANNs to learn new items by interleaving only a subset of old items with high similarity to the new item. We chose to retrieve the previously learned class exemplars based on their similarity with the new item, as the performance on highly similar old items will be most negatively impacted by new item learning. Using such similarity-weighted interleaved learning (SWIL), we could reach performance levels comparable to that achieved by using the entire training dataset, thereby reducing the amount of data required and learning time, making SWIL a fast and data-efficient approach. In addition, we recorded from the hippocampus and visual cortex of mice, while they were exploring virtual-reality contexts with different levels of similarity. We were able to demonstrate the same similarity-weighted (SWIL) replay of sequences in the hippocampus and visual cortex in our pilot experiments. We hope that the results presented here will shed light on new item learning and memory consolidation process in the brain and ANNs.
---------------
- YouTube link:
https://www.youtube.com/watch?v=kjUOhTsVl4g
- Microsoft Teams:
https://bit.ly/clai-seminars
- YouTube recordings of the previous sessions:
https://www.youtube.com/c/ContinualAI
---------------
Feel free to share this email to anyone interested and invite them to subscribe this mailing-list here: https://groups.google.com/g/continualai
Please also contact me if you want to speak at one of the next sessions!
Looking forward to seeing you all there!
All the best,
Keiland Cooper
University of California
ContinualAI Co-founder
http://kwcooper.xyz
Reply all
Reply to author
Forward
0 new messages