Trends in NeuroAI: fMRI foundation model via 4d Swin transformers
Paul Scotti
Kicking off the Trends in NeuroAI reading group! I hope this group brings together people enthusiastic about the latest developments in neuroimaging and AI and can be a hub for sharing, learning, and growing together regardless of prior experience.
Our meets will vary in format. Sometimes we'll have a guest speaker presenting their work (like our upcoming presentation tomorrow Nov 21 at 8:30am ET!) and other times, we'll engage in a more informal journal club style. Tomorrow's meeting covers a preprint that adapts the Swin transformer to 4d inputs, allowing for pre-training across large-scale fMRI datasets with downstream applications.
Meetings will be twice a month but exact dates and times are not fixed (it will be whatever is most convenient for the speaker), so hopefully if you can't make it to tomorrow's meeting that subsequent meetings will be at more convenient times.
Title:
SwiFT: Swin 4D fMRI Transformer
Abstract:
Modeling spatiotemporal brain dynamics from high-dimensional data, such as functional Magnetic Resonance Imaging (fMRI), is a formidable task in neuroscience. Existing approaches for fMRI analysis utilize hand-crafted features, but the process of feature extraction risks losing essential information in fMRI scans. To address this challenge, we present SwiFT (Swin 4D fMRI Transformer), a Swin Transformer architecture that can learn brain dynamics directly from fMRI volumes in a memory and computation-efficient manner. SwiFT achieves this by implementing a 4D window multi-head self-attention mechanism and absolute positional embeddings. We evaluate SwiFT using multiple large-scale resting-state fMRI datasets, including the Human Connectome Project (HCP), Adolescent Brain Cognitive Development (ABCD), and UK Biobank (UKB) datasets, to predict sex, age, and cognitive intelligence. Our experimental outcomes reveal that SwiFT consistently outperforms recent state-of-the-art models. Furthermore, by leveraging its end-to-end learning capability, we show that contrastive loss-based self-supervised pre-training of SwiFT can enhance performance on downstream tasks. Additionally, we employ an explainable AI method to identify the brain regions associated with sex classification. To our knowledge, SwiFT is the first Swin Transformer architecture to process dimensional spatiotemporal brain functional data in an end-to-end fashion. Our work holds substantial potential in facilitating scalable learning of functional brain imaging in neuroscience research by reducing the hurdles associated with applying Transformer models to high-dimensional fMRI.
Speaker:
Junbeom Kwon is a research associate working in Prof. Jiook Cha’s lab at Seoul National University.
Paper link: https://arxiv.org/abs/2307.05916
To access the Zoom link you can add the event to your Calendar (Google Calendar | iCal | View in browser)
Best,
Paul