Yao Hu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Yao Hu has a total of 27 co-authors. Here are the top co-authors by number of shared publications:
- Kay Chen Tan
The Hong Kong Polytechnic University
14 shared publications - Yu-An Huang
Northwestern Polytechnical University
10 shared publications - Rui Liu
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hung Hum, Hong Kong
8 shared publications - Linqi Song
City University of Hong Kong
5 shared publications - Xiaoming Xue
School of Petroleum Engineering, China University of Petroleum (East China), Qingdao, China
5 shared publications - Jibin Wu
The Hong Kong Polytechnic University
4 shared publications
Latest Publications
- Open Access
- 23 October 2026
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 42 (11)
AbLWR: a context-aware listwise ranking framework for antibody-antigen binding affinity prediction via positive-unlabeled learning
Motivation: Accurate prediction of antibody-antigen binding affinity is fundamental to therapeutic design and the rapid response to evolving pathogens. However, current computational approaches predominantly formulate this as a point-wise regression task. This paradigm struggles to discriminate subtle affinity differences among highly homologous variants and lacks the global context necessary to effectively prioritize top-tier candidates. Furthermore, the field is severely constrained by label sparsity. Although sequence data is abundant, experimentally verified affinities are scarce, leaving a massive pool of unverified antibody-antigen structural interactions unexploited by standard supervised paradigms. Results: We propose AbLWR (Antibody-antigen binding affinity Listwise Ranking), a novel framework that reformulates the conventional affinity regression task as a listwise ranking problem to directly optimize candidate prioritization. To mitigate label sparsity, AbLWR incorporates a Positive-Unlabeled (PU) learning mechanism, leveraging a dual-level contrastive objective and meta-optimized label refinement to distill robust structural representations from vast unlabeled data. To accommodate antigenic variation, we employ a homologous antigen sampling strategy coupled with a Multi-Head Self-Attention (MHSA) module, which explicitly models inter-sample dependencies within training lists to capture subtle affinity nuances. Extensive evaluations demonstrate that AbLWR significantly outperforms state-of-the-art sequence-, structure-, and complex-based baselines. It improves the Full Rank Accuracy (FRA) by approximately 8% compared to the strongest baseline in randomized cross-validation, and exhibits strong robustness in out-of-distribution scenarios involving distinct antigen and antibody clusters. Case studies on Influenza Viruses and Human Interleukin-33 further validate its practical utility. Specifically, AbLWR successfully tracks affinity drops caused by antigenic drift and immediately captures optimal binders in simulated wet-lab screening. Availability: Source code and datasets are freely available at https://github.com/Josie-xufan/AbLWR.
- 23 October 2026
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 42 (11)
Open AccessAbLWR: a context-aware listwise ranking framework for antibody-antigen binding affinity prediction via positive-unlabeled learning
Motivation: Accurate prediction of antibody-antigen binding affinity is fundamental to therapeutic design and the rapid response to evolving pathogens. However, current computational approaches predominantly formulate this as a point-wise regression task. This paradigm struggles to discriminate subtle affinity differences among highly homologous variants and lacks the global context necessary to effectively prioritize top-tier candidates. Furthermore, the field is severely constrained by label sparsity. Although sequence data is abundant, experimentally verified affinities are scarce, leaving a massive pool of unverified antibody-antigen structural interactions unexploited by standard supervised paradigms. Results: We propose AbLWR (Antibody-antigen binding affinity Listwise Ranking), a novel framework that reformulates the conventional affinity regression task as a listwise ranking problem to directly optimize candidate prioritization. To mitigate label sparsity, AbLWR incorporates a Positive-Unlabeled (PU) learning mechanism, leveraging a dual-level contrastive objective and meta-optimized label refinement to distill robust structural representations from vast unlabeled data. To accommodate antigenic variation, we employ a homologous antigen sampling strategy coupled with a Multi-Head Self-Attention (MHSA) module, which explicitly models inter-sample dependencies within training lists to capture subtle affinity nuances. Extensive evaluations demonstrate that AbLWR significantly outperforms state-of-the-art sequence-, structure-, and complex-based baselines. It improves the Full Rank Accuracy (FRA) by approximately 8% compared to the strongest baseline in randomized cross-validation, and exhibits strong robustness in out-of-distribution scenarios involving distinct antigen and antibody clusters. Case studies on Influenza Viruses and Human Interleukin-33 further validate its practical utility. Specifically, AbLWR successfully tracks affinity drops caused by antigenic drift and immediately captures optimal binders in simulated wet-lab screening. Availability: Source code and datasets are freely available at https://github.com/Josie-xufan/AbLWR.
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. PP (02780062) , 1
scBIT: Integrating Single-cell Transcriptomic Data into fMRI-based Prediction for Alzheimer's Disease Diagnosis
Functional MRI (fMRI) and single-cell transcri ptomics are pivotal in Alzheimer’s disease (AD) research, each providing unique insights into neural function and molecular mechanisms. However, integrating these complementary modalities remains largely unexplored. Here, we introduce scBIT, a novel method for enhancing AD prediction by combining fMRI with single-nucleus RNA (snRNA). scBIT leverages snRNA as an auxiliary modality, significantly improving fMRI-based prediction models and providing comprehensive interpretability. It employs a sampling strategy to segment snRNA data into cell-type-specific gene networks and utilizes a self-explainable graph neural network to extract critical subgraphs. Additionally, we use demographic and genetic similarities to pair snRNA and fMRI data across individuals, enabling robust cross-modal learning. Extensive experiments validate scBIT’s effectiveness in revealing intricate brain region-gene associations and enhancing diagnostic prediction accuracy. By advancing brain imaging transcriptomics to the single-cell level, scBIT sheds new light on biomarker discovery in AD research. Experimental results show that incorporating snRNA data into the scBIT model significantly boosts accuracy, improving binary classification by 3.39% and five-class classification by 26.59%. The codes were implemented in Python and have been released on GitHub (https://github.com/77YQ77/scBIT) and Zenodo (https://zenodo.org/records/11599030) with detailed instructions.
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. PP (02780062) , 1
scBIT: Integrating Single-cell Transcriptomic Data into fMRI-based Prediction for Alzheimer's Disease Diagnosis
Functional MRI (fMRI) and single-cell transcri ptomics are pivotal in Alzheimer’s disease (AD) research, each providing unique insights into neural function and molecular mechanisms. However, integrating these complementary modalities remains largely unexplored. Here, we introduce scBIT, a novel method for enhancing AD prediction by combining fMRI with single-nucleus RNA (snRNA). scBIT leverages snRNA as an auxiliary modality, significantly improving fMRI-based prediction models and providing comprehensive interpretability. It employs a sampling strategy to segment snRNA data into cell-type-specific gene networks and utilizes a self-explainable graph neural network to extract critical subgraphs. Additionally, we use demographic and genetic similarities to pair snRNA and fMRI data across individuals, enabling robust cross-modal learning. Extensive experiments validate scBIT’s effectiveness in revealing intricate brain region-gene associations and enhancing diagnostic prediction accuracy. By advancing brain imaging transcriptomics to the single-cell level, scBIT sheds new light on biomarker discovery in AD research. Experimental results show that incorporating snRNA data into the scBIT model significantly boosts accuracy, improving binary classification by 3.39% and five-class classification by 26.59%. The codes were implemented in Python and have been released on GitHub (https://github.com/77YQ77/scBIT) and Zenodo (https://zenodo.org/records/11599030) with detailed instructions.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (10) , 1-18
Enhancing X-ray Image Classification through Heterogeneous Federated Learning with Natural Image-Augmented Models
Deep learning-based computer-aided diagnosis (DL-CAD) models have achieved remarkable success in X-ray image analysis. Yet their effectiveness is often constrained by the tight regulations of governing sensitive X-ray data. Federated Learning (FL) offers a promising paradigm by facilitating the collaborative training of DL-CAD models across healthcare institutions without compromising data privacy. Despite this advancement, the limited local X-ray archives and the issue of heterogeneous model architectures bring distinctive challenges. To address these challenges, this work pioneers the utilization of natural images to develop a natural image-augmented heterogeneous FL framework (NatIMG-FL) for X-ray classification. For augmenting local training, NatIMG-FL leverages natural images as auxiliary supervised data to facilitate the alignment of feature distributions between natural and X-ray images. To tackle the model heterogeneity issue, NatIMG-FL introduces a novel dual weights-based fine-grained knowledge transfer method, enabling adaptive knowledge exchange between local and central models. The NatIMG-FL framework provides insights into exploiting natural images as proxy datasets to enhance knowledge transfer in heterogeneous FL for X-ray classification.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (10) , 1-18
Enhancing X-ray Image Classification through Heterogeneous Federated Learning with Natural Image-Augmented Models
Deep learning-based computer-aided diagnosis (DL-CAD) models have achieved remarkable success in X-ray image analysis. Yet their effectiveness is often constrained by the tight regulations of governing sensitive X-ray data. Federated Learning (FL) offers a promising paradigm by facilitating the collaborative training of DL-CAD models across healthcare institutions without compromising data privacy. Despite this advancement, the limited local X-ray archives and the issue of heterogeneous model architectures bring distinctive challenges. To address these challenges, this work pioneers the utilization of natural images to develop a natural image-augmented heterogeneous FL framework (NatIMG-FL) for X-ray classification. For augmenting local training, NatIMG-FL leverages natural images as auxiliary supervised data to facilitate the alignment of feature distributions between natural and X-ray images. To tackle the model heterogeneity issue, NatIMG-FL introduces a novel dual weights-based fine-grained knowledge transfer method, enabling adaptive knowledge exchange between local and central models. The NatIMG-FL framework provides insights into exploiting natural images as proxy datasets to enhance knowledge transfer in heterogeneous FL for X-ray classification.
- 8 May 2025
- Published by Elsevier in Medical Image Analysis
- Vol. 103, 103581
CausalMixNet: A mixed-attention framework for causal intervention in robust medical image diagnosis
- 8 May 2025
- Published by Elsevier in Medical Image Analysis
- Vol. 103, 103581
CausalMixNet: A mixed-attention framework for causal intervention in robust medical image diagnosis
- 4 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (3) , 1121-1134
Dynamic Graph Representation Learning for Spatio-Temporal Neuroimaging Analysis
Neuroimaging analysis aims to reveal the information-processing mechanisms of the human brain in a noninvasive manner. In the past, graph neural networks (GNNs) have shown promise in capturing the non-Euclidean structure of brain networks. However, existing neuroimaging studies focused primarily on spatial functional connectivity, despite temporal dynamics in complex brain networks. To address this gap, we propose a spatio-temporal interactive graph representation framework (STIGR) for dynamic neuroimaging analysis that encompasses different aspects from classification and regression tasks to interpretation tasks. STIGR leverages a dynamic adaptive-neighbor graph convolution network to capture the interrelationships between spatial and temporal dynamics. To address the limited global scope in graph convolutions, a self-attention module based on Transformers is introduced to extract long-term dependencies. Contrastive learning is used to adaptively contrast similarities between adjacent scanning windows, modeling cross-temporal correlations in dynamic graphs. Extensive experiments on six public neuroimaging datasets demonstrate the competitive performance of STIGR across different platforms, achieving state-of-the-art results in classification and regression tasks. The proposed framework enables the detection of remarkable temporal association patterns between regions of interest based on sequential neuroimaging signals, offering medical professionals a versatile and interpretable tool for exploring task-specific neurological patterns. Our codes and models are available at https://github.com/77YQ77/STIGR/.
- 4 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 55 (3) , 1121-1134
Dynamic Graph Representation Learning for Spatio-Temporal Neuroimaging Analysis
Neuroimaging analysis aims to reveal the information-processing mechanisms of the human brain in a noninvasive manner. In the past, graph neural networks (GNNs) have shown promise in capturing the non-Euclidean structure of brain networks. However, existing neuroimaging studies focused primarily on spatial functional connectivity, despite temporal dynamics in complex brain networks. To address this gap, we propose a spatio-temporal interactive graph representation framework (STIGR) for dynamic neuroimaging analysis that encompasses different aspects from classification and regression tasks to interpretation tasks. STIGR leverages a dynamic adaptive-neighbor graph convolution network to capture the interrelationships between spatial and temporal dynamics. To address the limited global scope in graph convolutions, a self-attention module based on Transformers is introduced to extract long-term dependencies. Contrastive learning is used to adaptively contrast similarities between adjacent scanning windows, modeling cross-temporal correlations in dynamic graphs. Extensive experiments on six public neuroimaging datasets demonstrate the competitive performance of STIGR across different platforms, achieving state-of-the-art results in classification and regression tasks. The proposed framework enables the detection of remarkable temporal association patterns between regions of interest based on sequential neuroimaging signals, offering medical professionals a versatile and interpretable tool for exploring task-specific neurological patterns. Our codes and models are available at https://github.com/77YQ77/STIGR/.
- 13 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (8) , 15055-15069
Anti-Confounding Hashing: Enhancing Radiological Image Retrieval via Debiased Weighting and Counterfactual Reasoning
Content-based medical image retrieval (CBMIR) enables physicians to make evidence-based diagnoses by retrieving similar medical images and recalling previous cases stored in databases. However, existing CBMIR models are prone to capturing superficial correlations due to confounding factors such as complex host organs and lesions, imaging discrepancies, artifacts, and inconsistent protocols. To address this issue, we propose a plug-and-play anti-confounding hashing (ACH) method, which uses debiased sample weighting and lesion counterfactual reasoning (LCR) to directly capture the natural direct effect (NDE) of lesions on query medical images without bias. The devised debiased weighting (DBW) loss adopts a backdoor adjustment to separate lesions from confounders. To effectively locate salient areas of lesions, we present a coarse-to-fine lesion positioning (C2F-LP) module by counterfactual reasoning. On two real-world radiological image datasets, ACH achieves 0.2%–9% improvement in mean average precision (mAP) over the six state-of-the-art methods, when using code lengths ranging from 8-bit to 32-bit. Its robustness to confounding factors is demonstrated through explainable visual analysis.
- 13 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (8) , 15055-15069
Anti-Confounding Hashing: Enhancing Radiological Image Retrieval via Debiased Weighting and Counterfactual Reasoning
Content-based medical image retrieval (CBMIR) enables physicians to make evidence-based diagnoses by retrieving similar medical images and recalling previous cases stored in databases. However, existing CBMIR models are prone to capturing superficial correlations due to confounding factors such as complex host organs and lesions, imaging discrepancies, artifacts, and inconsistent protocols. To address this issue, we propose a plug-and-play anti-confounding hashing (ACH) method, which uses debiased sample weighting and lesion counterfactual reasoning (LCR) to directly capture the natural direct effect (NDE) of lesions on query medical images without bias. The devised debiased weighting (DBW) loss adopts a backdoor adjustment to separate lesions from confounders. To effectively locate salient areas of lesions, we present a coarse-to-fine lesion positioning (C2F-LP) module by counterfactual reasoning. On two real-world radiological image datasets, ACH achieves 0.2%–9% improvement in mean average precision (mAP) over the six state-of-the-art methods, when using code lengths ranging from 8-bit to 32-bit. Its robustness to confounding factors is demonstrated through explainable visual analysis.
- 11 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 29 (6) , 2416-2430
Surrogate-Assisted Search With Competitive Knowledge Transfer for Expensive Optimization
Expensive optimization problems (EOPs) have attracted increasing research attention over the decades due to their ubiquity in a variety of practical applications. Despite many sophisticated surrogate-assisted evolutionary algorithms (SAEAs) that have been developed for solving such problems, most of them lack the ability to transfer knowledge from previously-solved tasks and always start their search from scratch, making them troubled by the notorious cold-start issue. A few preliminary studies that integrate transfer learning into SAEAs still face some issues, such as defective similarity quantification that is prone to underestimate promising knowledge, surrogate-dependency that makes the transfer methods not coherent with the state-of-the-art in SAEAs, etc. In light of the above, a plug and play competitive knowledge transfer method is proposed to boost various SAEAs in this paper. Specifically, both the optimized solutions from the source tasks and the promising solutions acquired by the target surrogate are treated as task-solving knowledge, enabling them to compete with each other to elect the winner for expensive evaluation, thus boosting the search speed on the target task. Moreover, the lower bound of the convergence gain brought by the knowledge competition is mathematically analyzed, which is expected to strengthen the theoretical foundation of sequential transfer optimization. Experimental studies conducted on a series of benchmark problems and a practical application from the petroleum industry verify the efficacy of the proposed method. The source code of the competitive knowledge transfer is available at https://github.com/XmingHsueh/SAS-CKT.
- 11 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 29 (6) , 2416-2430
Surrogate-Assisted Search With Competitive Knowledge Transfer for Expensive Optimization
Expensive optimization problems (EOPs) have attracted increasing research attention over the decades due to their ubiquity in a variety of practical applications. Despite many sophisticated surrogate-assisted evolutionary algorithms (SAEAs) that have been developed for solving such problems, most of them lack the ability to transfer knowledge from previously-solved tasks and always start their search from scratch, making them troubled by the notorious cold-start issue. A few preliminary studies that integrate transfer learning into SAEAs still face some issues, such as defective similarity quantification that is prone to underestimate promising knowledge, surrogate-dependency that makes the transfer methods not coherent with the state-of-the-art in SAEAs, etc. In light of the above, a plug and play competitive knowledge transfer method is proposed to boost various SAEAs in this paper. Specifically, both the optimized solutions from the source tasks and the promising solutions acquired by the target surrogate are treated as task-solving knowledge, enabling them to compete with each other to elect the winner for expensive evaluation, thus boosting the search speed on the target task. Moreover, the lower bound of the convergence gain brought by the knowledge competition is mathematically analyzed, which is expected to strengthen the theoretical foundation of sequential transfer optimization. Experimental studies conducted on a series of benchmark problems and a practical application from the petroleum industry verify the efficacy of the proposed method. The source code of the competitive knowledge transfer is available at https://github.com/XmingHsueh/SAS-CKT.
- 17 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (6) , 4046-4058
Spatio-Temporal Hybrid Attentive Graph Network for Diagnosis of Mental Disorders on fMRI Time-Series Data
Facing the prevalence of mental disorders around the world, the burden of healthcare services becomes increasingly imminent. To lessen patients' suffering, the timely diagnosis and therapy of mental disorders are particularly essential. Functional magnetic resonance imaging (fMRI), as the de facto non-invasive neuroimaging technique, can effectively examine the spatial and temporal patterns of brain activity. Recently, computer-aided diagnosis (CAD) approaches have emerged to assist doctors in interpreting fMRI images. However, existing CAD methods cannot fully exploit the spatio-temporal dependence in fMRI signals, possibly leading to inaccurate diagnosis. In this study, we propose a spatio-temporal hybrid attentive graph network (ST-HAG) for diagnosing autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) from fMRI data. Specifically, a hybrid graph convolution network is developed to effectively capture complex spatio-temporal dynamics. Meanwhile, a Transformer-based self-attention module helps ST-HAG to extract the full-scale temporal correlation. Finally, we use a gated fusion unit to learn discriminative spatio-temporal graph representations for classification. Cross-validation experiments demonstrate that the proposed ST-HAG achieves state-of-the-art performance with a mean accuracy of 71.9% and 74.8% for ASD and ADHD on ABIDE (1035 subjects) and ADHD-200 (939 subjects) datasets, respectively. Moreover, thanks to the adopted dynamic graph attentive representation, the potent interpretability enables ST-HAG to detect the remarkable temporal association patterns among different brain regions based on dynamic functional connectivity networks.
- 17 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (6) , 4046-4058
Spatio-Temporal Hybrid Attentive Graph Network for Diagnosis of Mental Disorders on fMRI Time-Series Data
Facing the prevalence of mental disorders around the world, the burden of healthcare services becomes increasingly imminent. To lessen patients' suffering, the timely diagnosis and therapy of mental disorders are particularly essential. Functional magnetic resonance imaging (fMRI), as the de facto non-invasive neuroimaging technique, can effectively examine the spatial and temporal patterns of brain activity. Recently, computer-aided diagnosis (CAD) approaches have emerged to assist doctors in interpreting fMRI images. However, existing CAD methods cannot fully exploit the spatio-temporal dependence in fMRI signals, possibly leading to inaccurate diagnosis. In this study, we propose a spatio-temporal hybrid attentive graph network (ST-HAG) for diagnosing autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) from fMRI data. Specifically, a hybrid graph convolution network is developed to effectively capture complex spatio-temporal dynamics. Meanwhile, a Transformer-based self-attention module helps ST-HAG to extract the full-scale temporal correlation. Finally, we use a gated fusion unit to learn discriminative spatio-temporal graph representations for classification. Cross-validation experiments demonstrate that the proposed ST-HAG achieves state-of-the-art performance with a mean accuracy of 71.9% and 74.8% for ASD and ADHD on ABIDE (1035 subjects) and ADHD-200 (939 subjects) datasets, respectively. Moreover, thanks to the adopted dynamic graph attentive representation, the potent interpretability enables ST-HAG to detect the remarkable temporal association patterns among different brain regions based on dynamic functional connectivity networks.
- 24 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-18
Source Free Semi-Supervised Transfer Learning for Diagnosis of Mental Disorders on fMRI Scans
The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Deep learning-based computer-aided diagnosis (CAD) has emerged to relieve the tension in healthcare institutions by detecting abnormal neuroimaging-derived phenotypes. However, training deep learning models relies on sufficient annotated datasets, which can be costly and laborious. Semi-supervised learning (SSL) and transfer learning (TL) can mitigate this challenge by leveraging unlabeled data within the same institution and advantageous information from source domain, respectively. This work is the first attempt to propose an effective semi-supervised transfer learning (SSTL) framework dubbed S3TL for CAD of mental disorders on fMRI data. Within S3TL, a secure cross-domain feature alignment method is developed to generate target-related source model in SSL. Subsequently, we propose an enhanced dual-stage pseudo-labeling approach to assign pseudo-labels for unlabeled samples in target domain. Finally, an advantageous knowledge transfer method is conducted to improve the generalization capability of the target model. Comprehensive experimental results demonstrate that S3TL achieves competitive accuracies of 69.14%, 69.65%, and 72.62% on ABIDE-I, ABIDE-II, and ADHD-200 datasets, respectively. Furthermore, the simulation experiments also demonstrate the application potential of S3TL through model interpretation analysis and federated learning extension.
- 24 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-18
Source Free Semi-Supervised Transfer Learning for Diagnosis of Mental Disorders on fMRI Scans
The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Deep learning-based computer-aided diagnosis (CAD) has emerged to relieve the tension in healthcare institutions by detecting abnormal neuroimaging-derived phenotypes. However, training deep learning models relies on sufficient annotated datasets, which can be costly and laborious. Semi-supervised learning (SSL) and transfer learning (TL) can mitigate this challenge by leveraging unlabeled data within the same institution and advantageous information from source domain, respectively. This work is the first attempt to propose an effective semi-supervised transfer learning (SSTL) framework dubbed S3TL for CAD of mental disorders on fMRI data. Within S3TL, a secure cross-domain feature alignment method is developed to generate target-related source model in SSL. Subsequently, we propose an enhanced dual-stage pseudo-labeling approach to assign pseudo-labels for unlabeled samples in target domain. Finally, an advantageous knowledge transfer method is conducted to improve the generalization capability of the target model. Comprehensive experimental results demonstrate that S3TL achieves competitive accuracies of 69.14%, 69.65%, and 72.62% on ABIDE-I, ABIDE-II, and ADHD-200 datasets, respectively. Furthermore, the simulation experiments also demonstrate the application potential of S3TL through model interpretation analysis and federated learning extension.
- 17 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10591-10605
Spatial–Temporal Co-Attention Learning for Diagnosis of Mental Disorders From Resting-State fMRI Data
Neuroimaging techniques have been widely adopted to detect the neurological brain structures and functions of the nervous system. As an effective noninvasive neuroimaging technique, functional magnetic resonance imaging (fMRI) has been extensively used in computer-aided diagnosis (CAD) of mental disorders, e.g., autism spectrum disorder (ASD) and attention deficit/hyperactivity disorder (ADHD). In this study, we propose a spatial–temporal co-attention learning (STCAL) model for diagnosing ASD and ADHD from fMRI data. In particular, a guided co-attention (GCA) module is developed to model the intermodal interactions of spatial and temporal signal patterns. A novel sliding cluster attention module is designed to address global feature dependency of self-attention mechanism in fMRI time series. Comprehensive experimental results demonstrate that our STCAL model can achieve competitive accuracies of 73.0 $\pm$ 4.5%, 72.0 $\pm$ 3.8%, and 72.5 $\pm$ 4.2% on the ABIDE I, ABIDE II, and ADHD-200 datasets, respectively. Moreover, the potential for feature pruning based on the co-attention scores is validated by the simulation experiment. The clinical interpretation analysis of STCAL can allow medical professionals to concentrate on the discriminative regions of interest and key time frames from fMRI data.
- 17 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (8) , 10591-10605
Spatial–Temporal Co-Attention Learning for Diagnosis of Mental Disorders From Resting-State fMRI Data
Neuroimaging techniques have been widely adopted to detect the neurological brain structures and functions of the nervous system. As an effective noninvasive neuroimaging technique, functional magnetic resonance imaging (fMRI) has been extensively used in computer-aided diagnosis (CAD) of mental disorders, e.g., autism spectrum disorder (ASD) and attention deficit/hyperactivity disorder (ADHD). In this study, we propose a spatial–temporal co-attention learning (STCAL) model for diagnosing ASD and ADHD from fMRI data. In particular, a guided co-attention (GCA) module is developed to model the intermodal interactions of spatial and temporal signal patterns. A novel sliding cluster attention module is designed to address global feature dependency of self-attention mechanism in fMRI time series. Comprehensive experimental results demonstrate that our STCAL model can achieve competitive accuracies of 73.0 $\pm$ 4.5%, 72.0 $\pm$ 3.8%, and 72.5 $\pm$ 4.2% on the ABIDE I, ABIDE II, and ADHD-200 datasets, respectively. Moreover, the potential for feature pruning based on the co-attention scores is validated by the simulation experiment. The clinical interpretation analysis of STCAL can allow medical professionals to concentrate on the discriminative regions of interest and key time frames from fMRI data.