Multi-Scale Spatio-Temporal Fusion With Adaptive Brain Topology Learning for fMRI Based Neural Decoding
- 23 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 28 (1) , 262-272
Abstract
Neural decoding aims to extract information from neurons' activities to reveal how the brain functions. Due to the inherent spatial and temporal characteristics of brain signals, spatio-temporal computing has become a hot topic for neural decoding. However, the extant spatio-temporal decoding methods usually use static brain topology, ignoring the dynamic patterns of the interaction between brain regions. Further, they do not identify the hierarchical organization of brain topology, leading to only superficial insight into brain spatio-temporal interactions. Therefore, here we propose a novel framework, the Multi-Scale Spatio-Temporal framework with Adaptive Brain Topology Learning (MSST-ABTL), for neural decoding. It includes two new capabilities to enhance spatio-temporal decoding: i) ABTL module, which learns dynamic brain topology while updating specific patterns of brain regions, ii) MSST module, which captures the association of spatial pattern and temporal evolution, and further enhances the interpretability of the learned dynamic topology from multi-scale perspective. We evaluated the framework on the public Human Connectome Project (HCP) dataset (resting-state and task-related fMRI data). The extensive experiments show that the proposed MSST-ABTL outperforms state-of-the-art methods on four evaluation metrics, and also can renew the neuroscientific discoveries in the brain's hierarchical patterns.Keywords
Funding Information
- Beijing Natural Science Foundation (4212037)
This publication has 65 references indexed in Scilit:
- The minimal preprocessing pipelines for the Human Connectome ProjectNeuroImage, 2013
- Dynamic connectivity regression: Determining state-related changes in brain connectivityNeuroImage, 2012
- Resting-state glutamate level in the anterior cingulate predicts blood-oxygen level-dependent response to cognitive controlProceedings of the National Academy of Sciences, 2012
- The organization of intrinsic brain activity differs between genders: A resting‐state fMRI study in a large cohort of young healthy subjectsHuman Brain Mapping, 2012
- Decoding Subject-Driven Cognitive States with Whole-Brain Connectivity PatternsCerebral Cortex, 2011
- Network structure of cerebral cortex shapes functional connectivity on multiple time scalesProceedings of the National Academy of Sciences, 2007
- Confirmation of region-specific patterns of gene expression in the human brainneurogenetics, 2007
- Directing the mind's eye: prefrontal, inferior and medial temporal mechanisms for visual working memoryCurrent Opinion in Neurobiology, 2005
- Functional magnetic resonance imaging (FMRI) of the human brainJournal of Neuroscience Methods, 1994
- A note on the stable decompostion of skew-symmetric matricesMathematics of Computation, 1982