Xintao Hu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xintao Hu has a total of 145 co-authors. Here are the top co-authors by number of shared publications:
- Tianming Liu
University of Georgia
13 shared publications Junwei Han
8 shared publications- Junwei Han
School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing, China
8 shared publications - Lei Guo
Northwestern Polytechnical University
8 shared publications Lei Guo
School of Automation, Northwestern Polytechnical University, Xi’an, China
7 shared publications- Lin Zhao
University of Georgia
6 shared publications
Latest Publications
- Top Cited Papers
- 1 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 73 (3) , 1050-1061
ChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Impression Generation on Multi-Institution and Multi-System Data
Achieving clinical level performance and widespread deployment for generating radiology impressions encounters a giant challenge for conventional artificial intelligence models tailored to specific diseases and organs. Concurrent with the increasing accessibility of radiology reports and advancements in modern general AI techniques, the emergence and potential of deployable radiology AI exploration have been bolstered. Here, we present ChatRadio-Valuer, the first general radiology diagnosis large language model for localized deployment within hospitals and being close to clinical use for multi-institution and multi-system diseases. ChatRadio-Valuer achieved 15 state-of-the-art results across five human systems and six institutions in clinical-level events (n = 332,673) through rigorous and full-spectrum assessment, including engineering metrics, clinical validation, and efficiency evaluation. Notably, it exceeded OpenAI's GPT-3.5 and GPT-4 models, achieving superior performance in comprehensive disease diagnosis compared to the average level of radiology experts. Besides, ChatRadio-Valuer supports zero-shot transfer learning, greatly boosting its effectiveness as a radiology assistant, while ensuring adherence to privacy standards and being readily utilized for large-scale patient populations. Our expeditions suggest the development of localized LLMs would become an imperative avenue in hospital applications.
- 1 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 73 (3) , 1050-1061
Top Cited PapersChatRadio-Valuer: A Chat Large Language Model for Generalizable Radiology Impression Generation on Multi-Institution and Multi-System Data
Achieving clinical level performance and widespread deployment for generating radiology impressions encounters a giant challenge for conventional artificial intelligence models tailored to specific diseases and organs. Concurrent with the increasing accessibility of radiology reports and advancements in modern general AI techniques, the emergence and potential of deployable radiology AI exploration have been bolstered. Here, we present ChatRadio-Valuer, the first general radiology diagnosis large language model for localized deployment within hospitals and being close to clinical use for multi-institution and multi-system diseases. ChatRadio-Valuer achieved 15 state-of-the-art results across five human systems and six institutions in clinical-level events (n = 332,673) through rigorous and full-spectrum assessment, including engineering metrics, clinical validation, and efficiency evaluation. Notably, it exceeded OpenAI's GPT-3.5 and GPT-4 models, achieving superior performance in comprehensive disease diagnosis compared to the average level of radiology experts. Besides, ChatRadio-Valuer supports zero-shot transfer learning, greatly boosting its effectiveness as a radiology assistant, while ensuring adherence to privacy standards and being readily utilized for large-scale patient populations. Our expeditions suggest the development of localized LLMs would become an imperative avenue in hospital applications.
- Open Access
FNF: Functional Network Fingerprint for Large Language Models
The development of large language models (LLMs) is costly and has significant commercial value. Consequently, preventing unauthorized appropriation of open-source LLMs and protecting developers' intellectual property rights have become critical challenges. In this work, we propose the Functional Network Fingerprint (FNF), a training-free, sample-efficient method for detecting whether a suspect LLM is derived from a victim model, based on the consistency between their functional network activity. We demonstrate that models that share a common origin, even with differences in scale or architecture, exhibit highly consistent patterns of neuronal activity within their functional networks across diverse input samples. In contrast, models trained independently on distinct data or with different objectives fail to preserve such activity alignment. Unlike conventional approaches, our method requires only a few samples for verification, preserves model utility, and remains robust to common model modifications (such as fine-tuning, pruning, and parameter permutation), as well as to comparisons across diverse architectures and dimensionalities. FNF thus provides model owners and third parties with a simple, non-invasive, and effective tool for protecting LLM intellectual property. The code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.
Open AccessFNF: Functional Network Fingerprint for Large Language Models
The development of large language models (LLMs) is costly and has significant commercial value. Consequently, preventing unauthorized appropriation of open-source LLMs and protecting developers' intellectual property rights have become critical challenges. In this work, we propose the Functional Network Fingerprint (FNF), a training-free, sample-efficient method for detecting whether a suspect LLM is derived from a victim model, based on the consistency between their functional network activity. We demonstrate that models that share a common origin, even with differences in scale or architecture, exhibit highly consistent patterns of neuronal activity within their functional networks across diverse input samples. In contrast, models trained independently on distinct data or with different objectives fail to preserve such activity alignment. Unlike conventional approaches, our method requires only a few samples for verification, preserves model utility, and remains robust to common model modifications (such as fine-tuning, pruning, and parameter permutation), as well as to comparisons across diverse architectures and dimensionalities. FNF thus provides model owners and third parties with a simple, non-invasive, and effective tool for protecting LLM intellectual property. The code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.
- Open Access
Jacobian-Based Interpretation of Nonlinear Neural Encoding Model
In recent years, the alignment between artificial neural network (ANN) embeddings and blood oxygenation level dependent (BOLD) responses in functional magnetic resonance imaging (fMRI) via neural encoding models has significantly advanced research on neural representation mechanisms and interpretability in the brain. However, these approaches remain limited in characterizing the brain's inherently nonlinear response properties. To address this, we propose the Jacobian-based Nonlinearity Evaluation (JNE), an interpretability metric for nonlinear neural encoding models. JNE quantifies nonlinearity by statistically measuring the dispersion of local linear mappings (Jacobians) from model representations to predicted BOLD responses, thereby approximating the nonlinearity of BOLD signals. Centered on proposing JNE as a novel interpretability metric, we validated its effectiveness through controlled simulation experiments on various activation functions and network architectures, and further verified it on real fMRI data, demonstrating a hierarchical progression of nonlinear characteristics from primary to higher-order visual cortices, consistent with established cortical organization. We further extended JNE with Sample-Specificity (JNE-SS), revealing stimulus-selective nonlinear response patterns in functionally specialized brain regions. As the first interpretability metric for quantifying nonlinear responses, JNE provides new insights into brain information processing. Code available at https://github.com/Gaitxh/JNE.
Open AccessJacobian-Based Interpretation of Nonlinear Neural Encoding Model
In recent years, the alignment between artificial neural network (ANN) embeddings and blood oxygenation level dependent (BOLD) responses in functional magnetic resonance imaging (fMRI) via neural encoding models has significantly advanced research on neural representation mechanisms and interpretability in the brain. However, these approaches remain limited in characterizing the brain's inherently nonlinear response properties. To address this, we propose the Jacobian-based Nonlinearity Evaluation (JNE), an interpretability metric for nonlinear neural encoding models. JNE quantifies nonlinearity by statistically measuring the dispersion of local linear mappings (Jacobians) from model representations to predicted BOLD responses, thereby approximating the nonlinearity of BOLD signals. Centered on proposing JNE as a novel interpretability metric, we validated its effectiveness through controlled simulation experiments on various activation functions and network architectures, and further verified it on real fMRI data, demonstrating a hierarchical progression of nonlinear characteristics from primary to higher-order visual cortices, consistent with established cortical organization. We further extended JNE with Sample-Specificity (JNE-SS), revealing stimulus-selective nonlinear response patterns in functionally specialized brain regions. As the first interpretability metric for quantifying nonlinear responses, JNE provides new insights into brain information processing. Code available at https://github.com/Gaitxh/JNE.
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (11) , 8361-8373
A Foundational fMRI Model for Representing Continuous Brain States
Foundational models have significant potential to advance brain function research, particularly in understanding the dynamics of brain states. However, most existing models process brain signals within fixed time windows, restricting their ability to capture the full temporal complexity of brain activity. In this study, we propose BrainSN (Brain States Network), a novel fMRI foundational model designed to represent continuous brain state information and support diverse downstream tasks. First, leveraging a transformer-based architecture, BrainSN reconstructs input brain states across multiple time scales and predicts future brain activity, effectively capturing both short-term and long-term dependencies. Second, through multiple embeddings and a channel gating module, the model integrates brain state information and applies an attention mechanism to extract critical features. Additionally, we train BrainSN on 1,256 hours of resting-state and naturalistic stimulus fMRI data, enabling it to learn large-scale brain dynamics without relying on task-based paradigms. Without fine-tuning, BrainSN achieves 75.23% and 75.82% accuracy in autism and attention disorder diagnosis tasks, respectively, matching the performance of leading models pretrained on disease-specific data. After fine-tuning, it surpasses these models. In mental state decoding, BrainSN attains 95.31% accuracy without fine-tuning, outperforming the best models trained on large-scale task-based fMRI data. Furthermore, by analyzing BrainSN's embeddings in relation to movie stimuli, we demonstrate that the model effectively captures the semantic content of movie scenes embedded in fMRI signals and is highly sensitive to sequence. These results highlight BrainSN's ability to model brain state dynamics and underscore its potential advantages for clinical diagnosis, treatment evaluation, and cognitive neuroscience research.
- 13 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (11) , 8361-8373
A Foundational fMRI Model for Representing Continuous Brain States
Foundational models have significant potential to advance brain function research, particularly in understanding the dynamics of brain states. However, most existing models process brain signals within fixed time windows, restricting their ability to capture the full temporal complexity of brain activity. In this study, we propose BrainSN (Brain States Network), a novel fMRI foundational model designed to represent continuous brain state information and support diverse downstream tasks. First, leveraging a transformer-based architecture, BrainSN reconstructs input brain states across multiple time scales and predicts future brain activity, effectively capturing both short-term and long-term dependencies. Second, through multiple embeddings and a channel gating module, the model integrates brain state information and applies an attention mechanism to extract critical features. Additionally, we train BrainSN on 1,256 hours of resting-state and naturalistic stimulus fMRI data, enabling it to learn large-scale brain dynamics without relying on task-based paradigms. Without fine-tuning, BrainSN achieves 75.23% and 75.82% accuracy in autism and attention disorder diagnosis tasks, respectively, matching the performance of leading models pretrained on disease-specific data. After fine-tuning, it surpasses these models. In mental state decoding, BrainSN attains 95.31% accuracy without fine-tuning, outperforming the best models trained on large-scale task-based fMRI data. Furthermore, by analyzing BrainSN's embeddings in relation to movie stimuli, we demonstrate that the model effectively captures the semantic content of movie scenes embedded in fMRI signals and is highly sensitive to sequence. These results highlight BrainSN's ability to model brain state dynamics and underscore its potential advantages for clinical diagnosis, treatment evaluation, and cognitive neuroscience research.
- Open Access
LinBridge: A Learnable Framework for Interpreting Nonlinear Neural Encoding Models
Neural encoding of artificial neural networks (ANNs) links their computational representations to brain responses, offering insights into how the brain processes information. Current studies mostly use linear encoding models for clarity, even though brain responses are often nonlinear. This has sparked interest in developing nonlinear encoding models that are still interpretable. To address this problem, we propose LinBridge, a learnable and flexible framework based on Jacobian analysis for interpreting nonlinear encoding models. LinBridge posits that the nonlinear mapping between ANN representations and neural responses can be factorized into a linear inherent component that approximates the complex nonlinear relationship, and a mapping bias that captures sample-selective nonlinearity. The Jacobian matrix, which reflects output change rates relative to input, enables the analysis of sample-selective mapping in nonlinear models. LinBridge employs a self-supervised learning strategy to extract both the linear inherent component and nonlinear mapping biases from the Jacobian matrices of the test set, allowing it to adapt effectively to various nonlinear encoding models. We validate the LinBridge framework in the scenario of neural visual encoding, using computational visual representations from CLIP-ViT to predict brain activity recorded via functional magnetic resonance imaging (fMRI). Our experimental results demonstrate that: 1) the linear inherent component extracted by LinBridge accurately reflects the complex mappings of nonlinear neural encoding models; 2) the sample-selective mapping bias elucidates the variability of nonlinearity across different levels of the visual processing hierarchy. This study presents a novel tool for interpreting nonlinear neural encoding models and offers fresh evidence about hierarchical nonlinearity distribution in the visual cortex.
Open AccessLinBridge: A Learnable Framework for Interpreting Nonlinear Neural Encoding Models
Neural encoding of artificial neural networks (ANNs) links their computational representations to brain responses, offering insights into how the brain processes information. Current studies mostly use linear encoding models for clarity, even though brain responses are often nonlinear. This has sparked interest in developing nonlinear encoding models that are still interpretable. To address this problem, we propose LinBridge, a learnable and flexible framework based on Jacobian analysis for interpreting nonlinear encoding models. LinBridge posits that the nonlinear mapping between ANN representations and neural responses can be factorized into a linear inherent component that approximates the complex nonlinear relationship, and a mapping bias that captures sample-selective nonlinearity. The Jacobian matrix, which reflects output change rates relative to input, enables the analysis of sample-selective mapping in nonlinear models. LinBridge employs a self-supervised learning strategy to extract both the linear inherent component and nonlinear mapping biases from the Jacobian matrices of the test set, allowing it to adapt effectively to various nonlinear encoding models. We validate the LinBridge framework in the scenario of neural visual encoding, using computational visual representations from CLIP-ViT to predict brain activity recorded via functional magnetic resonance imaging (fMRI). Our experimental results demonstrate that: 1) the linear inherent component extracted by LinBridge accurately reflects the complex mappings of nonlinear neural encoding models; 2) the sample-selective mapping bias elucidates the variability of nonlinearity across different levels of the visual processing hierarchy. This study presents a novel tool for interpreting nonlinear neural encoding models and offers fresh evidence about hierarchical nonlinearity distribution in the visual cortex.
- 14 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 11191-11203
BI-AVAN: A Brain-Inspired Adversarial Visual Attention Network for Characterizing Human Visual Attention From Neural Activity
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks. However, most of the visual attention studies adopted eye-tracking data rather than the direct measurement of brain activity to characterize human visual attention. In addition, the adversarial relationship between the attention-related objects and attention-neglected background in the human visual system was not fully exploited. To bridge these gaps, we propose a novel brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our BI-AVAN model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner. We use independent eye-tracking data as ground truth for validation and experimental results show that our model achieves robust and promising results when inferring meaningful human visual attention and mapping the relationship between brain activities and visual stimuli. Our BI-AVAN model contributes to the emerging field of leveraging the brain's functional architecture to inspire and guide the model design in artificial intelligence (AI), e.g., deep neural networks.
- 14 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 11191-11203
BI-AVAN: A Brain-Inspired Adversarial Visual Attention Network for Characterizing Human Visual Attention From Neural Activity
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks. However, most of the visual attention studies adopted eye-tracking data rather than the direct measurement of brain activity to characterize human visual attention. In addition, the adversarial relationship between the attention-related objects and attention-neglected background in the human visual system was not fully exploited. To bridge these gaps, we propose a novel brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our BI-AVAN model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner. We use independent eye-tracking data as ground truth for validation and experimental results show that our model achieves robust and promising results when inferring meaningful human visual attention and mapping the relationship between brain activities and visual stimuli. Our BI-AVAN model contributes to the emerging field of leveraging the brain's functional architecture to inspire and guide the model design in artificial intelligence (AI), e.g., deep neural networks.
- 6 March 2024
- Published by Elsevier in Medical Image Analysis
- Vol. 94, 103136
Task sub-type states decoding via group deep bidirectional recurrent neural network
- 6 March 2024
- Published by Elsevier in Medical Image Analysis
- Vol. 94, 103136
Task sub-type states decoding via group deep bidirectional recurrent neural network
- 9 January 2024
- Published by Springer Nature in Brain Structure and Function
- Vol. 229 (2) , 431-442
Frequency-specific functional difference between gyri and sulci in naturalistic paradigm fMRI
Disentangling functional difference between cortical folding patterns of gyri and sulci provides novel insights into the relationship between brain structure and function. Previous studies using resting-state functional magnetic resonance imaging (rsfMRI) have revealed that sulcal signals exhibit stronger high-frequency but weaker low-frequency components compared to gyral ones, suggesting that gyri may serve as functional integration centers while sulci are segregated local processing units. In this study, we utilize naturalistic paradigm fMRI (nfMRI) to explore the functional difference between gyri and sulci as it has proven to record stronger functional integrations compared to rsfMRI. We adopt a convolutional neural network (CNN) to classify gyral and sulcal fMRI signals in the whole brain (the global model) and within functional brain networks (the local models). The frequency-specific difference between gyri and sulci is then inferred from the power spectral density (PSD) profiles of the learned filters in the CNN model. Our experimental results show that nfMRI shows higher gyral–sulcal PSD contrast effect sizes in the global model compared to rsfMRI. In the local models, the effect sizes are either increased or decreased depending on frequency bands and functional complexity of the FBNs. This study highlights the advantages of nfMRI in depicting the functional difference between gyri and sulci, and provides novel insights into unraveling the relationship between brain structure and function.
- 9 January 2024
- Published by Springer Nature in Brain Structure and Function
- Vol. 229 (2) , 431-442
Frequency-specific functional difference between gyri and sulci in naturalistic paradigm fMRI
Disentangling functional difference between cortical folding patterns of gyri and sulci provides novel insights into the relationship between brain structure and function. Previous studies using resting-state functional magnetic resonance imaging (rsfMRI) have revealed that sulcal signals exhibit stronger high-frequency but weaker low-frequency components compared to gyral ones, suggesting that gyri may serve as functional integration centers while sulci are segregated local processing units. In this study, we utilize naturalistic paradigm fMRI (nfMRI) to explore the functional difference between gyri and sulci as it has proven to record stronger functional integrations compared to rsfMRI. We adopt a convolutional neural network (CNN) to classify gyral and sulcal fMRI signals in the whole brain (the global model) and within functional brain networks (the local models). The frequency-specific difference between gyri and sulci is then inferred from the power spectral density (PSD) profiles of the learned filters in the CNN model. Our experimental results show that nfMRI shows higher gyral–sulcal PSD contrast effect sizes in the global model compared to rsfMRI. In the local models, the effect sizes are either increased or decreased depending on frequency bands and functional complexity of the FBNs. This study highlights the advantages of nfMRI in depicting the functional difference between gyri and sulci, and provides novel insights into unraveling the relationship between brain structure and function.
- 6 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 71 (5) , 1442-1453
MEET: A Multi-Band EEG Transformer for Brain States Decoding
Objective: Electroencephalography (EEG) is among the most widely used and inexpensive neuroimaging techniques. Compared to the CNN or RNN based models, Transformer can better capture the temporal information in EEG signals and focus more on global features of the brain's functional activities. Importantly, according to the multiscale nature of EEG signals, it is crucial to consider the multi-band concept into the design of EEG Transformer architecture. Methods: We propose a novel M ulti-band EE G T ransformer (MEET) to represent and analyze the multiscale temporal time series of human brain EEG signals. MEET mainly includes three parts: 1) transform the EEG signals into multi-band images, and preserve the 3D spatial information between electrodes; 2) design a Band Attention Block to compute the attention maps of the stacked multi-band images and infer the fused feature maps; 3) apply the Temporal Self-Attention and Spatial Self-Attention modules to extract the spatiotemporal features for the characterization and differentiation of multi-frame dynamic brain states. Results: The experimental results show that: 1) MEET outperforms state-of-the-art methods on multiple open EEG datasets (SEED, SEED-IV, WM) for brain states classification; 2) MEET demonstrates that 5-bands fusion is the best integration strategy; and 3) MEET identifies interpretable brain attention regions. Significance: MEET is an interpretable and universal model based on the multiband-multiscale characteristics of EEG. Conclusion: The innovative combination of band attention and temporal/spatial self-attention mechanisms in MEET achieves promising data-driven learning of the temporal dependencies and spatial relationships of EEG signals across the entire brain in a holistic and comprehensive fashion.
- 6 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 71 (5) , 1442-1453
MEET: A Multi-Band EEG Transformer for Brain States Decoding
Objective: Electroencephalography (EEG) is among the most widely used and inexpensive neuroimaging techniques. Compared to the CNN or RNN based models, Transformer can better capture the temporal information in EEG signals and focus more on global features of the brain's functional activities. Importantly, according to the multiscale nature of EEG signals, it is crucial to consider the multi-band concept into the design of EEG Transformer architecture. Methods: We propose a novel M ulti-band EE G T ransformer (MEET) to represent and analyze the multiscale temporal time series of human brain EEG signals. MEET mainly includes three parts: 1) transform the EEG signals into multi-band images, and preserve the 3D spatial information between electrodes; 2) design a Band Attention Block to compute the attention maps of the stacked multi-band images and infer the fused feature maps; 3) apply the Temporal Self-Attention and Spatial Self-Attention modules to extract the spatiotemporal features for the characterization and differentiation of multi-frame dynamic brain states. Results: The experimental results show that: 1) MEET outperforms state-of-the-art methods on multiple open EEG datasets (SEED, SEED-IV, WM) for brain states classification; 2) MEET demonstrates that 5-bands fusion is the best integration strategy; and 3) MEET identifies interpretable brain attention regions. Significance: MEET is an interpretable and universal model based on the multiband-multiscale characteristics of EEG. Conclusion: The innovative combination of band attention and temporal/spatial self-attention mechanisms in MEET achieves promising data-driven learning of the temporal dependencies and spatial relationships of EEG signals across the entire brain in a holistic and comprehensive fashion.