Eloy Geenjaar
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Eloy Geenjaar has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Vince D. Calhoun
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (Georgia State University, Georgia Institute of Technology, Emory University), Atlanta, USA
8 shared publications Donghyun Kim
Affiliation Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory, Atlanta, Georgia, United States of America
3 shared publications- Lei Wu
Tri-institutional center for Translational Research in Neuroimaging and Data Science (TReNDS) Center, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, Georgia 30303
3 shared publications Sergey M. Plis
Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory, Atlanta, GA, USA
3 shared publicationsVince Calhoun
Affiliations School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia, United States of America, Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory, Atlanta, Georgia, United States of America
3 shared publications- Xinhui Li
Tri-Institutional Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, United States
3 shared publications
Latest Publications
- Open Access
- 29 July 2026
- Published by Springer Nature in Communications Biology
A scalable subject-specific manifold learning framework for neuroimaging data
Low-dimensional manifolds underlie mental and cognitive representations, and characterizing individual differences is critical to neuropsychiatric research. Existing manifold learning methods either distort subject-specific features through population-level alignment or limit cross-subject comparisons with individualized models. We introduce a scalable, subject-specific manifold learning framework with two modules: one capturing individual spatial variation via individualized spatial weights, and an efficient variant reducing computational cost with minimal performance loss. Simulations show that population-level alignment distorts manifold geometry, whereas our approach preserves subject-specific structure. Applied to movie-watching fMRI, our modules improve behavioral label recovery and reconstruction accuracy. In clinical resting-state fMRI, the efficient module identifies individualized spatial activation patterns highlighting sensorimotor, learning, and emotion-related regions. Further validation on an independent resting-state dataset demonstrates 95% test-retest identification accuracy. Together, these results show our framework is robust, reproducible, and broadly applicable across computational neuroscience and clinical research.
- 29 July 2026
- Published by Springer Nature in Communications Biology
Open AccessA scalable subject-specific manifold learning framework for neuroimaging data
Low-dimensional manifolds underlie mental and cognitive representations, and characterizing individual differences is critical to neuropsychiatric research. Existing manifold learning methods either distort subject-specific features through population-level alignment or limit cross-subject comparisons with individualized models. We introduce a scalable, subject-specific manifold learning framework with two modules: one capturing individual spatial variation via individualized spatial weights, and an efficient variant reducing computational cost with minimal performance loss. Simulations show that population-level alignment distorts manifold geometry, whereas our approach preserves subject-specific structure. Applied to movie-watching fMRI, our modules improve behavioral label recovery and reconstruction accuracy. In clinical resting-state fMRI, the efficient module identifies individualized spatial activation patterns highlighting sensorimotor, learning, and emotion-related regions. Further validation on an independent resting-state dataset demonstrates 95% test-retest identification accuracy. Together, these results show our framework is robust, reproducible, and broadly applicable across computational neuroscience and clinical research.
- Open Access
- 5 May 2026
- Published by MDPI AG in Preprints.org
A Systematic Review of Data-Driven Multimodal Fusion Methods for Simultaneous EEG-fMRI
Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide complementary views of brain activity, capturing neural dynamics across temporal and spatial scales. Integrating these modalities offers a powerful approach for studying brain function, yet remains fundamentally challenging due to differences in measurement mechanisms, temporal resolution, and neurovascular coupling. At its core, EEG–fMRI fusion can be viewed as an inverse problem: the goal is to recover latent neural processes that are only partially observed through electrophysiological and hemodynamic signals. Here, we review data-driven fusion methods developed between 2000 and 2025, focusing on approaches that aim to identify shared neural representations across modalities. We organize the existing methods according to the fusion strategy (symmetric vs. asymmetric), the methodological objective (factorization vs. translation), and the modeling assumptions (linear vs. non-linear), and discuss commonly-used evaluation metrics and visualization strategies. We further examine evaluation strategies, highlighting the lack of a universal validation standard and the challenges of interpreting latent multimodal components. Across neurological, psychiatric, and cognitive applications, EEG-fMRI fusion has revealed distributed network dynamics that are not accessible through unimodal analyses. However, key challenges remain, including temporal misalignment, noise-induced coupling, and model-dependent interpretation. We discuss emerging directions such as nonlinear modeling, flexible coupling frameworks, and large-scale group-level fusion, which may enable more robust and interpretable multimodal integration. Together, this review reframes EEG-fMRI fusion as a problem of latent neural inference and outlines a path toward more principled, scalable, and biologically grounded approaches for understanding brain function and dysfunction.
- 5 May 2026
- Published by MDPI AG in Preprints.org
Open AccessA Systematic Review of Data-Driven Multimodal Fusion Methods for Simultaneous EEG-fMRI
Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) provide complementary views of brain activity, capturing neural dynamics across temporal and spatial scales. Integrating these modalities offers a powerful approach for studying brain function, yet remains fundamentally challenging due to differences in measurement mechanisms, temporal resolution, and neurovascular coupling. At its core, EEG–fMRI fusion can be viewed as an inverse problem: the goal is to recover latent neural processes that are only partially observed through electrophysiological and hemodynamic signals. Here, we review data-driven fusion methods developed between 2000 and 2025, focusing on approaches that aim to identify shared neural representations across modalities. We organize the existing methods according to the fusion strategy (symmetric vs. asymmetric), the methodological objective (factorization vs. translation), and the modeling assumptions (linear vs. non-linear), and discuss commonly-used evaluation metrics and visualization strategies. We further examine evaluation strategies, highlighting the lack of a universal validation standard and the challenges of interpreting latent multimodal components. Across neurological, psychiatric, and cognitive applications, EEG-fMRI fusion has revealed distributed network dynamics that are not accessible through unimodal analyses. However, key challenges remain, including temporal misalignment, noise-induced coupling, and model-dependent interpretation. We discuss emerging directions such as nonlinear modeling, flexible coupling frameworks, and large-scale group-level fusion, which may enable more robust and interpretable multimodal integration. Together, this review reframes EEG-fMRI fusion as a problem of latent neural inference and outlines a path toward more principled, scalable, and biologically grounded approaches for understanding brain function and dysfunction.
- Open Access
- 27 April 2026
- Published by MIT Press in Imaging Neuroscience
- Vol. 4
Brain functional network connectivity interpolation characterizes the neuropsychiatric continuum and heterogeneity
Psychiatric and neurodevelopmental disorders such as schizophrenia (SZ) and autism spectrum disorder (ASD) are challenging to characterize in part due to their heterogeneous presentation in individuals, with symptoms now believed to exist on a continuum. Conventional diagnostic and neuroimaging analytical approaches rely on subjective assessment or group differences, but typically ignore progression between groups or heterogeneity within a group. To estimate the neuropsychiatric continuum and heterogeneity, we proposed a functional network connectivity (FNC) interpolation framework based on a variational autoencoder (VAE) using static FNC (sFNC) and dynamic FNC (dFNC) data from controls and patients with SZ or ASD. We demonstrated that VAEs significantly outperformed a linear baseline and a semi-supervised counterpart. For both sFNC and dFNC interpolation, the generated results effectively captured representative and generalizable properties in the original data. The interpolated continua from controls to patients in both SZ and ASD revealed group-wise gradients characterized by reduced positive correlations within the auditory, sensorimotor, and visual networks, as well as between the subcortical and cerebellar domains. In contrast, anti-correlations weakened between the subcortical domain and the sensory domains, and between the cerebellar domain and the sensory domains. Finally, we showed examples of how to generate continuous FNC data following group- or state-based trajectories in the VAE latent space. The proposed framework offers added advantages over traditional methods, including data-driven discovery of hidden relationships, visualization of individual differences, imputation of missing values along a continuous spectrum, and estimation of the stage where an individual falls within the continuum.
- 27 April 2026
- Published by MIT Press in Imaging Neuroscience
- Vol. 4
Open AccessBrain functional network connectivity interpolation characterizes the neuropsychiatric continuum and heterogeneity
Psychiatric and neurodevelopmental disorders such as schizophrenia (SZ) and autism spectrum disorder (ASD) are challenging to characterize in part due to their heterogeneous presentation in individuals, with symptoms now believed to exist on a continuum. Conventional diagnostic and neuroimaging analytical approaches rely on subjective assessment or group differences, but typically ignore progression between groups or heterogeneity within a group. To estimate the neuropsychiatric continuum and heterogeneity, we proposed a functional network connectivity (FNC) interpolation framework based on a variational autoencoder (VAE) using static FNC (sFNC) and dynamic FNC (dFNC) data from controls and patients with SZ or ASD. We demonstrated that VAEs significantly outperformed a linear baseline and a semi-supervised counterpart. For both sFNC and dFNC interpolation, the generated results effectively captured representative and generalizable properties in the original data. The interpolated continua from controls to patients in both SZ and ASD revealed group-wise gradients characterized by reduced positive correlations within the auditory, sensorimotor, and visual networks, as well as between the subcortical and cerebellar domains. In contrast, anti-correlations weakened between the subcortical domain and the sensory domains, and between the cerebellar domain and the sensory domains. Finally, we showed examples of how to generate continuous FNC data following group- or state-based trajectories in the VAE latent space. The proposed framework offers added advantages over traditional methods, including data-driven discovery of hidden relationships, visualization of individual differences, imputation of missing values along a continuous spectrum, and estimation of the stage where an individual falls within the continuum.
- Open Access
- 12 June 2025
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 20 (6) , e0324066
Providing context: Extracting non-linear and dynamic temporal motifs from brain activity
Approaches studying the dynamics of resting-state functional magnetic resonance imaging (rs-fMRI) activity often focus on time-resolved functional connectivity (tr-FC). While many tr-FC approaches have been proposed, most are linear approaches, e.g. computing the linear correlation at a timestep or within a window. In this work, we propose to use a generative non-linear deep learning model, a disentangled variational autoencoder (DSVAE), that factorizes out window-specific (context) information from timestep-specific (local) information. This has the advantage of allowing our model to capture differences at multiple temporal scales. We find that by separating out temporal scales our model’s window-specific embeddings, or as we refer to them, context embeddings, more accurately separate windows from schizophrenia patients and control subjects than baseline models and the standard tr-FC approach in a low-dimensional space. Moreover, we find that for individuals with schizophrenia, our model’s context embedding space is significantly correlated with both age and symptom severity. Interestingly, patients appear to spend more time in three clusters, one closer to controls which shows increased visual-sensorimotor, cerebellar-subcortical, and reduced cerebellar-visual functional network connectivity (FNC), an intermediate station showing increased subcortical-sensorimotor FNC, and one that shows decreased visual-sensorimotor, decreased subcortical-sensorimotor, and increased visual-subcortical domains. We verify that our model captures features that are complementary to - but not the same as - standard tr-FC features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics.
- 12 June 2025
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 20 (6) , e0324066
Open AccessProviding context: Extracting non-linear and dynamic temporal motifs from brain activity
Approaches studying the dynamics of resting-state functional magnetic resonance imaging (rs-fMRI) activity often focus on time-resolved functional connectivity (tr-FC). While many tr-FC approaches have been proposed, most are linear approaches, e.g. computing the linear correlation at a timestep or within a window. In this work, we propose to use a generative non-linear deep learning model, a disentangled variational autoencoder (DSVAE), that factorizes out window-specific (context) information from timestep-specific (local) information. This has the advantage of allowing our model to capture differences at multiple temporal scales. We find that by separating out temporal scales our model’s window-specific embeddings, or as we refer to them, context embeddings, more accurately separate windows from schizophrenia patients and control subjects than baseline models and the standard tr-FC approach in a low-dimensional space. Moreover, we find that for individuals with schizophrenia, our model’s context embedding space is significantly correlated with both age and symptom severity. Interestingly, patients appear to spend more time in three clusters, one closer to controls which shows increased visual-sensorimotor, cerebellar-subcortical, and reduced cerebellar-visual functional network connectivity (FNC), an intermediate station showing increased subcortical-sensorimotor FNC, and one that shows decreased visual-sensorimotor, decreased subcortical-sensorimotor, and increased visual-subcortical domains. We verify that our model captures features that are complementary to - but not the same as - standard tr-FC features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics.
- Open Access
Mapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging data
Mental and cognitive representations are believed to reside on low-dimensional, non-linear manifolds embedded within high-dimensional brain activity. Uncovering these manifolds is key to understanding individual differences in brain function, yet most existing machine learning methods either rely on population-level spatial alignment or assume data that is temporally structured, either because data is aligned among subjects or because event timings are known. We introduce a manifold learning framework that can capture subject-specific spatial variations across both structured and temporally unstructured neuroimaging data. On simulated data and two naturalistic fMRI datasets (Sherlock and Forrest Gump), our framework outperforms group-based baselines by recovering more accurate and individualized representations. We further show that the framework scales efficiently to large datasets and generalizes well to new subjects. To test this, we apply the framework to temporally unstructured resting-state fMRI data from individuals with schizophrenia and healthy controls. We further apply our method to a large resting-state fMRI dataset comprising individuals with schizophrenia and controls. In this setting, we demonstrate that the framework scales efficiently to large populations and generalizes robustly to unseen subjects. The learned subject-specific spatial maps our model finds reveal clinically relevant patterns, including increased activation in the basal ganglia, visual, auditory, and somatosensory regions, and decreased activation in the insula, inferior frontal gyrus, and angular gyrus. These findings suggest that our framework can uncover clinically relevant subject-specific brain activity patterns. Our approach thus provides a scalable and individualized framework for modeling brain activity, with applications in computational neuroscience and clinical research.
Open AccessMapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging data
Mental and cognitive representations are believed to reside on low-dimensional, non-linear manifolds embedded within high-dimensional brain activity. Uncovering these manifolds is key to understanding individual differences in brain function, yet most existing machine learning methods either rely on population-level spatial alignment or assume data that is temporally structured, either because data is aligned among subjects or because event timings are known. We introduce a manifold learning framework that can capture subject-specific spatial variations across both structured and temporally unstructured neuroimaging data. On simulated data and two naturalistic fMRI datasets (Sherlock and Forrest Gump), our framework outperforms group-based baselines by recovering more accurate and individualized representations. We further show that the framework scales efficiently to large datasets and generalizes well to new subjects. To test this, we apply the framework to temporally unstructured resting-state fMRI data from individuals with schizophrenia and healthy controls. We further apply our method to a large resting-state fMRI dataset comprising individuals with schizophrenia and controls. In this setting, we demonstrate that the framework scales efficiently to large populations and generalizes robustly to unseen subjects. The learned subject-specific spatial maps our model finds reveal clinically relevant patterns, including increased activation in the basal ganglia, visual, auditory, and somatosensory regions, and decreased activation in the insula, inferior frontal gyrus, and angular gyrus. These findings suggest that our framework can uncover clinically relevant subject-specific brain activity patterns. Our approach thus provides a scalable and individualized framework for modeling brain activity, with applications in computational neuroscience and clinical research.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (16) , 16781-16789
CiTrus: Squeezing Extra Performance out of Low-data Bio-signal Transfer Learning
Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (16) , 16781-16789
Open AccessCiTrus: Squeezing Extra Performance out of Low-data Bio-signal Transfer Learning
Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.
- Open Access
CiTrus: Squeezing Extra Performance out of Low-data Bio-signal Transfer Learning
Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.
Open AccessCiTrus: Squeezing Extra Performance out of Low-data Bio-signal Transfer Learning
Transfer learning for bio-signals has recently become an important technique to improve prediction performance on downstream tasks with small bio-signal datasets. Recent works have shown that pre-training a neural network model on a large dataset (e.g. EEG) with a self-supervised task, replacing the self-supervised head with a linear classification head, and fine-tuning the model on different downstream bio-signal datasets (e.g., EMG or ECG) can dramatically improve the performance on those datasets. In this paper, we propose a new convolution-transformer hybrid model architecture with masked auto-encoding for low-data bio-signal transfer learning, introduce a frequency-based masked auto-encoding task, employ a more comprehensive evaluation framework, and evaluate how much and when (multimodal) pre-training improves fine-tuning performance. We also introduce a dramatically more performant method of aligning a downstream dataset with a different temporal length and sampling rate to the original pre-training dataset. Our findings indicate that the convolution-only part of our hybrid model can achieve state-of-the-art performance on some low-data downstream tasks. The performance is often improved even further with our full model. In the case of transformer-based models we find that pre-training especially improves performance on downstream datasets, multimodal pre-training often increases those gains further, and our frequency-based pre-training performs the best on average for the lowest and highest data regimes.
- Open Access
Brain functional network connectivity interpolation characterizes the neuropsychiatric continuum and heterogeneity
Psychiatric and neurodevelopmental disorders such as schizophrenia (SZ) and autism spectrum disorder (ASD) are challenging to characterize in part due to their heterogeneous presentation in individuals, with symptoms now believed to exist on a continuum. Conventional diagnostic and neuroimaging analytical approaches rely on subjective assessment or group differences but typically ignore progression between groups or heterogeneity within a group. To estimate the neuropsychiatric continuum and heterogeneity, we propose a functional network connectivity (FNC) interpolation framework based on a variational autoencoder (VAE) using static FNC (sFNC) and dynamic FNC (dFNC) data from controls and patients with SZ or ASD. We demonstrate that VAEs significantly outperform a linear baseline and a semi-supervised counterpart. For both sFNC and dFNC interpolation, the generated results effectively capture representative and generalizable properties in the original data. The interpolated continua from controls to patients in both disorders reveal group-wise gradients characterized by reduced positive correlations within the auditory, sensorimotor, and visual networks, as well as between the subcortical and cerebellar domains. In contrast, anti-correlations weaken between the subcortical domain and sensory domains, and between the cerebellar domain and sensory regions. Finally, we show examples of how to generate continuous FNC data following pathological or state-based trajectories in the VAE latent space. The proposed framework offers added advantages over traditional methods, including data-driven discovery of hidden relationships, visualization of individual differences, imputation of missing values along a continuous spectrum, and estimation of the stage where an individual falls within the continuum.
Open AccessBrain functional network connectivity interpolation characterizes the neuropsychiatric continuum and heterogeneity
Psychiatric and neurodevelopmental disorders such as schizophrenia (SZ) and autism spectrum disorder (ASD) are challenging to characterize in part due to their heterogeneous presentation in individuals, with symptoms now believed to exist on a continuum. Conventional diagnostic and neuroimaging analytical approaches rely on subjective assessment or group differences but typically ignore progression between groups or heterogeneity within a group. To estimate the neuropsychiatric continuum and heterogeneity, we propose a functional network connectivity (FNC) interpolation framework based on a variational autoencoder (VAE) using static FNC (sFNC) and dynamic FNC (dFNC) data from controls and patients with SZ or ASD. We demonstrate that VAEs significantly outperform a linear baseline and a semi-supervised counterpart. For both sFNC and dFNC interpolation, the generated results effectively capture representative and generalizable properties in the original data. The interpolated continua from controls to patients in both disorders reveal group-wise gradients characterized by reduced positive correlations within the auditory, sensorimotor, and visual networks, as well as between the subcortical and cerebellar domains. In contrast, anti-correlations weaken between the subcortical domain and sensory domains, and between the cerebellar domain and sensory regions. Finally, we show examples of how to generate continuous FNC data following pathological or state-based trajectories in the VAE latent space. The proposed framework offers added advantages over traditional methods, including data-driven discovery of hidden relationships, visualization of individual differences, imputation of missing values along a continuous spectrum, and estimation of the stage where an individual falls within the continuum.
- Open Access
Providing context: Extracting non-linear and dynamic temporal motifs from brain activity
Approaches studying the dynamics of resting-state functional magnetic resonance imaging (rs-fMRI) activity often focus on time-resolved functional connectivity (tr-FC). While many approaches have been proposed, these typically focus on linear approaches like computing the linear correlation at a timestep or within a window. In this work, we propose to use a generative non-linear deep learning model, a disentangled variational autoencoder (DSVAE), that factorizes out window-specific (context) information from timestep-specific (local) information. This has the advantage of allowing our model to capture differences at multiple temporal scales. For the timestep-specific scale, which has higher temporal precision, we find significant differences between schizophrenia patients and control subjects in their temporal step distance through our model’s latent space. We also find that window-specific embeddings, or as we refer to them, context embeddings, more accurately separate windows from schizophrenia patients and control subjects than the standard tr-FC approach. Moreover, we find that for individuals with schizophrenia, our model’s context embedding space is significantly correlated with both age and symptom severity. Interestingly, patients appear to spend more time in three clusters, one closer to controls which shows increased visual-sensorimotor, cerebellar-subcortical, and reduced cerebellar-sensorimotor functional network connectivity (FNC), an intermediate station showing increased subcortical-sensorimotor FNC, and one that shows decreased visual-sensorimotor, decreased subcortical-sensorimotor, and increased visual-subcortical domains. We verify that our model captures features that are complementary to - but not the same as - standard tr-FC features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics. Author summary We introduce a powerful new temporal representation learning model for resting-state fMRI data that can find novel and interesting temporal motifs that relate brain activity to schizophrenia diagnosis. Our model uses factorizes out local (timestep-specific) and context (window-specific) information in a latent space. We find that the context embeddings are more linearly predictive of whether someone has a schizophrenia diagnosis, and the context embedding space significantly correlates with age and symptom severity. We verify that our model captures features that are complementary to but not the same as time-resolved functional connectivity features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics.
Open AccessProviding context: Extracting non-linear and dynamic temporal motifs from brain activity
Approaches studying the dynamics of resting-state functional magnetic resonance imaging (rs-fMRI) activity often focus on time-resolved functional connectivity (tr-FC). While many approaches have been proposed, these typically focus on linear approaches like computing the linear correlation at a timestep or within a window. In this work, we propose to use a generative non-linear deep learning model, a disentangled variational autoencoder (DSVAE), that factorizes out window-specific (context) information from timestep-specific (local) information. This has the advantage of allowing our model to capture differences at multiple temporal scales. For the timestep-specific scale, which has higher temporal precision, we find significant differences between schizophrenia patients and control subjects in their temporal step distance through our model’s latent space. We also find that window-specific embeddings, or as we refer to them, context embeddings, more accurately separate windows from schizophrenia patients and control subjects than the standard tr-FC approach. Moreover, we find that for individuals with schizophrenia, our model’s context embedding space is significantly correlated with both age and symptom severity. Interestingly, patients appear to spend more time in three clusters, one closer to controls which shows increased visual-sensorimotor, cerebellar-subcortical, and reduced cerebellar-sensorimotor functional network connectivity (FNC), an intermediate station showing increased subcortical-sensorimotor FNC, and one that shows decreased visual-sensorimotor, decreased subcortical-sensorimotor, and increased visual-subcortical domains. We verify that our model captures features that are complementary to - but not the same as - standard tr-FC features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics. Author summary We introduce a powerful new temporal representation learning model for resting-state fMRI data that can find novel and interesting temporal motifs that relate brain activity to schizophrenia diagnosis. Our model uses factorizes out local (timestep-specific) and context (window-specific) information in a latent space. We find that the context embeddings are more linearly predictive of whether someone has a schizophrenia diagnosis, and the context embedding space significantly correlates with age and symptom severity. We verify that our model captures features that are complementary to but not the same as time-resolved functional connectivity features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics.
- Open Access
- 16 December 2023
- Published by Elsevier in NeuroImage
- Vol. 285, 120485
Self-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links
- 16 December 2023
- Published by Elsevier in NeuroImage
- Vol. 285, 120485
Open AccessSelf-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links