Xu-Yao Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Xu-Yao Zhang has a total of 154 co-authors. Here are the top co-authors by number of shared publications:
- Cheng-Lin Liu
Chinese Academy of Sciences
31 shared publications - Fei Zhu
Chinese Academy of Sciences
14 shared publications - Da-Han Wang
Xiamen University of Technology
12 shared publications - Zhen Cheng
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
9 shared publications - Shunzhi Zhu
Xiamen University of Technology, Xiamen, China
7 shared publications Cheng-Lin Liu
6 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 4279-4288
Semantic-Spatial Discriminability Enhancement for Generalized Visual Grounding
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 4279-4288
Open AccessSemantic-Spatial Discriminability Enhancement for Generalized Visual Grounding
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (11) , 15267-15286
The Hitchhiker’s Guide to Autonomous Research: A Survey of Scientific Agents
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (11) , 15267-15286
The Hitchhiker’s Guide to Autonomous Research: A Survey of Scientific Agents
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (11) , 15143-15160
Happy++: Toward Stable and Unified Continual Generalized Category Discovery
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (11) , 15143-15160
Happy++: Toward Stable and Unified Continual Generalized Category Discovery
- Open Access
- 26 September 2026
- Published by Elsevier in SSRN Electronic Journal
QE-Catalytic: A Graph–Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption
Relaxed energy is a core physical descriptor of catalytic reactivity, and its prediction accuracy directly determines the reliability of machine-learning-driven catalyst screening. While $E(3)$-equivariant graph neural networks (GNNs) can effectively model three-dimensional atomic structures and achieve strong performance, language-model-based approaches offer human-readable representations and reduce reliance on explicit graphs, but remain limited in energy-prediction accuracy and discriminating multiple configurations within the same system, even with GAP-CatBERTa-style graph-assisted pretraining. To address these limitations, we propose \textbf{QE-Catalytic}, a multimodal framework that tightly integrates a large language model (Qwen) with an $E(3)$-equivariant graph Transformer (Equiformer-V2) for relaxed adsorption-energy prediction and inverse design on complex catalytic surfaces. Across multiple benchmarks and evaluation metrics, QE-Catalytic consistently outperforms baseline models. Our code is available at https://github.com/1716757342/QE-Catalytic
- 26 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessQE-Catalytic: A Graph–Language Multimodal Base Model for Relaxed-Energy Prediction in Catalytic Adsorption
Relaxed energy is a core physical descriptor of catalytic reactivity, and its prediction accuracy directly determines the reliability of machine-learning-driven catalyst screening. While $E(3)$-equivariant graph neural networks (GNNs) can effectively model three-dimensional atomic structures and achieve strong performance, language-model-based approaches offer human-readable representations and reduce reliance on explicit graphs, but remain limited in energy-prediction accuracy and discriminating multiple configurations within the same system, even with GAP-CatBERTa-style graph-assisted pretraining. To address these limitations, we propose \textbf{QE-Catalytic}, a multimodal framework that tightly integrates a large language model (Qwen) with an $E(3)$-equivariant graph Transformer (Equiformer-V2) for relaxed adsorption-energy prediction and inverse design on complex catalytic surfaces. Across multiple benchmarks and evaluation metrics, QE-Catalytic consistently outperforms baseline models. Our code is available at https://github.com/1716757342/QE-Catalytic
- Open Access
- 24 September 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e90870
Automated Brief Hospital Course Summarization in Cardiac Surgery Using a Lightweight Large Language Model–Based Framework: Development and Evaluation Study on the Medical Information Mart for Intensive Care-IV
Background: Physician documentation requirements are a known contributor to clinician burnout, with the manual creation of brief hospital course (BHC) summaries being particularly time-consuming. Automating BHC summarization may mitigate this workload and reduce documentation errors. However, current natural language processing (NLP) methods are often limited to single-document inputs, and large language models (LLMs) face privacy and deployment challenges. Furthermore, existing methods often require manual information extraction and struggle to maintain temporal accuracy. Objective: We developed and evaluated LiteMedDoc, a lightweight, locally deployable LLM-based framework to automatically generate BHC summaries without model fine-tuning. Our objective was to determine whether clinically useful clinical summaries could be generated under strict privacy and resource constraints, making automated summarization feasible in real-world hospital environments. Methods: LiteMedDoc is a modular pipeline built upon an 8-billion-parameter open-source LLM (Llama 3.1). It features 3 specialized modules: a static dynamic information hierarchy module to condense multisource inputs and structure clinical events chronologically; a similar document retrieval-augmented generation module that retrieves contextually relevant prior case summaries; and a self-adaptive feedback optimization module used offline for prompt optimization. All processing was performed locally without any model fine-tuning. We evaluated the framework on a retrospective cohort of 4538 coronary artery bypass grafting (CABG) surgery cases from the Medical Information Mart for Intensive Care (MIMIC)-IV database. A held-out test set of 403 cases was used to generate BHC summaries. The model-generated summaries were compared to reference BHCs using 8 standard NLP metrics covering lexical overlap (BLEU-4 [Bilingual Evaluation Understudy-4 gram] and ROUGE [Recall-Oriented Understudy for Gisting Evaluation]), semantic similarity (BERTScore and METEOR [Metric for Evaluation of Translation With Explicit Ordering]), and clinical relevance (AlignScore [Alignment Score] and MEDCON [Medical Concept Overlap]). Additionally, 15 cardiac surgeons conducted a clinical evaluation of a sample of model-generated summaries, rating them on completeness, correctness, readability, conciseness, and global quality using a 5-point Likert scale. Results: Without any model training, LiteMedDoc achieved strong performance across individual automated metrics, nearly matching a fine-tuned model and exceeding a 70-billion-parameter model on all metrics. Additionally, in a within-database cross-domain evaluation on lobectomy cases, the framework maintained encouraging performance after prompt adaptation and outperformed both the base model and the CABG-fine-tuned model. Surgeons rated the AI-generated summaries above the prespecified acceptability threshold across all domains (mean scores ≥3.0), specifically praising their structure and conciseness. Conclusions: By integrating 3 specialized modules, the proposed framework offers a practical, locally deployable solution for clinician-in-the-loop BHC draft generation under privacy and resource constraints, and holds promise for improving documentation efficiency and enhancing information continuity during care transitions.
- 24 September 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e90870
Open AccessAutomated Brief Hospital Course Summarization in Cardiac Surgery Using a Lightweight Large Language Model–Based Framework: Development and Evaluation Study on the Medical Information Mart for Intensive Care-IV
Background: Physician documentation requirements are a known contributor to clinician burnout, with the manual creation of brief hospital course (BHC) summaries being particularly time-consuming. Automating BHC summarization may mitigate this workload and reduce documentation errors. However, current natural language processing (NLP) methods are often limited to single-document inputs, and large language models (LLMs) face privacy and deployment challenges. Furthermore, existing methods often require manual information extraction and struggle to maintain temporal accuracy. Objective: We developed and evaluated LiteMedDoc, a lightweight, locally deployable LLM-based framework to automatically generate BHC summaries without model fine-tuning. Our objective was to determine whether clinically useful clinical summaries could be generated under strict privacy and resource constraints, making automated summarization feasible in real-world hospital environments. Methods: LiteMedDoc is a modular pipeline built upon an 8-billion-parameter open-source LLM (Llama 3.1). It features 3 specialized modules: a static dynamic information hierarchy module to condense multisource inputs and structure clinical events chronologically; a similar document retrieval-augmented generation module that retrieves contextually relevant prior case summaries; and a self-adaptive feedback optimization module used offline for prompt optimization. All processing was performed locally without any model fine-tuning. We evaluated the framework on a retrospective cohort of 4538 coronary artery bypass grafting (CABG) surgery cases from the Medical Information Mart for Intensive Care (MIMIC)-IV database. A held-out test set of 403 cases was used to generate BHC summaries. The model-generated summaries were compared to reference BHCs using 8 standard NLP metrics covering lexical overlap (BLEU-4 [Bilingual Evaluation Understudy-4 gram] and ROUGE [Recall-Oriented Understudy for Gisting Evaluation]), semantic similarity (BERTScore and METEOR [Metric for Evaluation of Translation With Explicit Ordering]), and clinical relevance (AlignScore [Alignment Score] and MEDCON [Medical Concept Overlap]). Additionally, 15 cardiac surgeons conducted a clinical evaluation of a sample of model-generated summaries, rating them on completeness, correctness, readability, conciseness, and global quality using a 5-point Likert scale. Results: Without any model training, LiteMedDoc achieved strong performance across individual automated metrics, nearly matching a fine-tuned model and exceeding a 70-billion-parameter model on all metrics. Additionally, in a within-database cross-domain evaluation on lobectomy cases, the framework maintained encouraging performance after prompt adaptation and outperformed both the base model and the CABG-fine-tuned model. Surgeons rated the AI-generated summaries above the prespecified acceptability threshold across all domains (mean scores ≥3.0), specifically praising their structure and conciseness. Conclusions: By integrating 3 specialized modules, the proposed framework offers a practical, locally deployable solution for clinician-in-the-loop BHC draft generation under privacy and resource constraints, and holds promise for improving documentation efficiency and enhancing information continuity during care transitions.
- Open Access
- 6 May 2026
- Published by MDPI AG in Mathematics
- Vol. 14 (9) , 1568
STAGE: LLM-Driven Semantic and Topological Augmented Graph Embedding for Text-Attributed Graphs
Text-attributed graphs (TAGs) require models to jointly exploit node text and graph structure, yet doing so effectively remains difficult when node text is sparse and the structural context is large. Here, we propose STAGE (Semantic and Topological Augmented Graph Embedding), a two-stage framework for representation learning on TAGs. In Stage I, a frozen large language model is used offline to generate explanatory text that enriches compressed node attributes without introducing online LLM training cost. In Stage II, STAGE performs structure-aware representation learning under a fixed global token budget by combining random-walk-based structural context with graph-conditioned token reduction before PLM encoding. This design preserves informative semantic content while preventing unconstrained sequence expansion. Experiments on seven benchmark datasets show that STAGE consistently outperforms strong baselines under the same evaluation setting and maintains favorable efficiency under bounded input-length constraints.
- 6 May 2026
- Published by MDPI AG in Mathematics
- Vol. 14 (9) , 1568
Open AccessSTAGE: LLM-Driven Semantic and Topological Augmented Graph Embedding for Text-Attributed Graphs
Text-attributed graphs (TAGs) require models to jointly exploit node text and graph structure, yet doing so effectively remains difficult when node text is sparse and the structural context is large. Here, we propose STAGE (Semantic and Topological Augmented Graph Embedding), a two-stage framework for representation learning on TAGs. In Stage I, a frozen large language model is used offline to generate explanatory text that enriches compressed node attributes without introducing online LLM training cost. In Stage II, STAGE performs structure-aware representation learning under a fixed global token budget by combining random-walk-based structural context with graph-conditioned token reduction before PLM encoding. This design preserves informative semantic content while preventing unconstrained sequence expansion. Experiments on seven benchmark datasets show that STAGE consistently outperforms strong baselines under the same evaluation setting and maintains favorable efficiency under bounded input-length constraints.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-13
Video-Level Cross-Modal Temporal-Navigation for RGBT Tracking
RGBT tracking has recently garnered significant attention due to its all-weather tracking capability. Traditional RGBT tracking methods primarily concentrate on the fusion of cross-modal spatial information. However, these methods ignore contextual relationships between consecutive video frames and lack effective interactions between modalities, easily resulting in tracking drift gradually due to the accumulation of errors. To avoid this limitation, we propose a novel Video-Level Cross-Modal Temporal-Navigation method termed VCT for robust RGBT tracking, which fully leverages the complementary spatio-temporal information across modalities to improve cross-modal tracking accuracy. The VCT employs a simple, flexible, and effective video-level dual-stream architecture that accommodates video sequences of arbitrary length, enabling RGB and TIR streams to capture and synergize spatio-temporal features across frames. To achieve temporal consistency and adaptability, we design a Cross-Modal Temporal Prompt Navigator (CM-TPN) that dynamically aggregates and compresses the historical frame context to navigate predictions of subsequent frames through temporal prompts. In addition, we introduce a Modality-Specific Mixture of Adapters (MS-MoA) to promote the spatio-temporal interaction both within and between modalities, thereby dramatically adapting to appearance changes. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the four popular RGBT tracking benchmarks.
- 27 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-13
Video-Level Cross-Modal Temporal-Navigation for RGBT Tracking
RGBT tracking has recently garnered significant attention due to its all-weather tracking capability. Traditional RGBT tracking methods primarily concentrate on the fusion of cross-modal spatial information. However, these methods ignore contextual relationships between consecutive video frames and lack effective interactions between modalities, easily resulting in tracking drift gradually due to the accumulation of errors. To avoid this limitation, we propose a novel Video-Level Cross-Modal Temporal-Navigation method termed VCT for robust RGBT tracking, which fully leverages the complementary spatio-temporal information across modalities to improve cross-modal tracking accuracy. The VCT employs a simple, flexible, and effective video-level dual-stream architecture that accommodates video sequences of arbitrary length, enabling RGB and TIR streams to capture and synergize spatio-temporal features across frames. To achieve temporal consistency and adaptability, we design a Cross-Modal Temporal Prompt Navigator (CM-TPN) that dynamically aggregates and compresses the historical frame context to navigate predictions of subsequent frames through temporal prompts. In addition, we introduce a Modality-Specific Mixture of Adapters (MS-MoA) to promote the spatio-temporal interaction both within and between modalities, thereby dramatically adapting to appearance changes. Extensive experiments demonstrate that our method achieves state-of-the-art performance on the four popular RGBT tracking benchmarks.
- 28 January 2026
- Published by Springer Nature
- p. 17-31
Dual-Branch Residual Wavelet Attention Network for Colorectal Cancer Magnifying Endoscopy Image Classification
- 28 January 2026
- Published by Springer Nature
- p. 17-31
Dual-Branch Residual Wavelet Attention Network for Colorectal Cancer Magnifying Endoscopy Image Classification
- 15 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-15
Debias Fine-Grained Radical Modeling via Global-Local Attention Mutual Learning for Zero-Shot Chinese Character Recognition
In zero-shot Chinese character recognition (ZSCCR), radicals play a critical bridge for transferring semantic knowledge from seen to unseen characters. However, existing radical-based approaches tend to coarsely map Chinese character images to radical embeddings, disregarding the imperative for fine-grained radical learning and the prevalent imbalanced issue among radicals. This leads to underemphasizing less frequent radicals or introducing bias towards more prevalent ones. To address this, we propose a novel framework of Debiasing fine-grained Radical Modeling via Global-Local radical attention mutual learning for ZSCCR, termed DeRMGL. DeRMGL features two branches for global and local radical modeling. The multi-local branch partitions learnable radical prototypes into non-overlapping blocks by frequency, each with a dedicated attention network. Since radicals in each block have similar occurrence frequencies, attention scores are less skewed by frequency disparities, effectively alleviating radical imbalance. The global branch maintains full radical coverage to coordinate holistic semantic scoring. During training, mutual learning loss optimizes both branches collaboratively, and during inference, their predictions are combined. Furthermore, an enhanced Radical Prototype-based Attention (RPA) network serves as the backbone to leverage visual-semantic knowledge. Extensive experiments on four ZSCCR benchmark datasets demonstrate that DeRMGL achieves state-of-the-art performance.
- 15 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. PP (15209210) , 1-15
Debias Fine-Grained Radical Modeling via Global-Local Attention Mutual Learning for Zero-Shot Chinese Character Recognition
In zero-shot Chinese character recognition (ZSCCR), radicals play a critical bridge for transferring semantic knowledge from seen to unseen characters. However, existing radical-based approaches tend to coarsely map Chinese character images to radical embeddings, disregarding the imperative for fine-grained radical learning and the prevalent imbalanced issue among radicals. This leads to underemphasizing less frequent radicals or introducing bias towards more prevalent ones. To address this, we propose a novel framework of Debiasing fine-grained Radical Modeling via Global-Local radical attention mutual learning for ZSCCR, termed DeRMGL. DeRMGL features two branches for global and local radical modeling. The multi-local branch partitions learnable radical prototypes into non-overlapping blocks by frequency, each with a dedicated attention network. Since radicals in each block have similar occurrence frequencies, attention scores are less skewed by frequency disparities, effectively alleviating radical imbalance. The global branch maintains full radical coverage to coordinate holistic semantic scoring. During training, mutual learning loss optimizes both branches collaboratively, and during inference, their predictions are combined. Furthermore, an enhanced Radical Prototype-based Attention (RPA) network serves as the backbone to leverage visual-semantic knowledge. Extensive experiments on four ZSCCR benchmark datasets demonstrate that DeRMGL achieves state-of-the-art performance.
- Open Access
- 5 January 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Automated Brief Hospital Course Summarization in Cardiac Surgery Using a Lightweight Large Language Model–Based Framework: Development and Evaluation Study on the Medical Information Mart for Intensive Care-IV (Preprint)
BACKGROUND Physician documentation requirements are a known contributor to clinician burnout, with the manual creation of brief hospital course (BHC) summaries being particularly time-consuming. Automating BHC summarization may mitigate this workload and reduce documentation errors. However, current natural language processing (NLP) methods are often limited to single-document inputs, and large language models (LLMs) face privacy and deployment challenges. Furthermore, existing methods often require manual information extraction and struggle to maintain temporal accuracy. OBJECTIVE We developed and evaluated LiteMedDoc, a lightweight, locally deployable LLM-based framework to automatically generate BHC summaries without model fine-tuning. Our objective was to determine whether clinically useful clinical summaries could be generated under strict privacy and resource constraints, making automated summarization feasible in real-world hospital environments. METHODS LiteMedDoc is a modular pipeline built upon an 8-billion-parameter open-source LLM (Llama 3.1). It features 3 specialized modules: a static dynamic information hierarchy module to condense multisource inputs and structure clinical events chronologically; a similar document retrieval-augmented generation module that retrieves contextually relevant prior case summaries; and a self-adaptive feedback optimization module used offline for prompt optimization. All processing was performed locally without any model fine-tuning. We evaluated the framework on a retrospective cohort of 4538 coronary artery bypass grafting (CABG) surgery cases from the Medical Information Mart for Intensive Care (MIMIC)-IV database. A held-out test set of 403 cases was used to generate BHC summaries. The model-generated summaries were compared to reference BHCs using 8 standard NLP metrics covering lexical overlap (BLEU-4 [Bilingual Evaluation Understudy-4 gram] and ROUGE [Recall-Oriented Understudy for Gisting Evaluation]), semantic similarity (BERTScore and METEOR [Metric for Evaluation of Translation With Explicit Ordering]), and clinical relevance (AlignScore [Alignment Score] and MEDCON [Medical Concept Overlap]). Additionally, 15 cardiac surgeons conducted a clinical evaluation of a sample of model-generated summaries, rating them on completeness, correctness, readability, conciseness, and global quality using a 5-point Likert scale. RESULTS Without any model training, LiteMedDoc achieved strong performance across individual automated metrics, nearly matching a fine-tuned model and exceeding a 70-billion-parameter model on all metrics. Additionally, in a within-database cross-domain evaluation on lobectomy cases, the framework maintained encouraging performance after prompt adaptation and outperformed both the base model and the CABG-fine-tuned model. Surgeons rated the AI-generated summaries above the prespecified acceptability threshold across all domains (mean scores ≥3.0), specifically praising their structure and conciseness. CONCLUSIONS By integrating 3 specialized modules, the proposed framework offers a practical, locally deployable solution for clinician-in-the-loop BHC draft generation under privacy and resource constraints, and holds promise for improving documentation efficiency and enhancing information continuity during care transitions.
- 5 January 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAutomated Brief Hospital Course Summarization in Cardiac Surgery Using a Lightweight Large Language Model–Based Framework: Development and Evaluation Study on the Medical Information Mart for Intensive Care-IV (Preprint)
BACKGROUND Physician documentation requirements are a known contributor to clinician burnout, with the manual creation of brief hospital course (BHC) summaries being particularly time-consuming. Automating BHC summarization may mitigate this workload and reduce documentation errors. However, current natural language processing (NLP) methods are often limited to single-document inputs, and large language models (LLMs) face privacy and deployment challenges. Furthermore, existing methods often require manual information extraction and struggle to maintain temporal accuracy. OBJECTIVE We developed and evaluated LiteMedDoc, a lightweight, locally deployable LLM-based framework to automatically generate BHC summaries without model fine-tuning. Our objective was to determine whether clinically useful clinical summaries could be generated under strict privacy and resource constraints, making automated summarization feasible in real-world hospital environments. METHODS LiteMedDoc is a modular pipeline built upon an 8-billion-parameter open-source LLM (Llama 3.1). It features 3 specialized modules: a static dynamic information hierarchy module to condense multisource inputs and structure clinical events chronologically; a similar document retrieval-augmented generation module that retrieves contextually relevant prior case summaries; and a self-adaptive feedback optimization module used offline for prompt optimization. All processing was performed locally without any model fine-tuning. We evaluated the framework on a retrospective cohort of 4538 coronary artery bypass grafting (CABG) surgery cases from the Medical Information Mart for Intensive Care (MIMIC)-IV database. A held-out test set of 403 cases was used to generate BHC summaries. The model-generated summaries were compared to reference BHCs using 8 standard NLP metrics covering lexical overlap (BLEU-4 [Bilingual Evaluation Understudy-4 gram] and ROUGE [Recall-Oriented Understudy for Gisting Evaluation]), semantic similarity (BERTScore and METEOR [Metric for Evaluation of Translation With Explicit Ordering]), and clinical relevance (AlignScore [Alignment Score] and MEDCON [Medical Concept Overlap]). Additionally, 15 cardiac surgeons conducted a clinical evaluation of a sample of model-generated summaries, rating them on completeness, correctness, readability, conciseness, and global quality using a 5-point Likert scale. RESULTS Without any model training, LiteMedDoc achieved strong performance across individual automated metrics, nearly matching a fine-tuned model and exceeding a 70-billion-parameter model on all metrics. Additionally, in a within-database cross-domain evaluation on lobectomy cases, the framework maintained encouraging performance after prompt adaptation and outperformed both the base model and the CABG-fine-tuned model. Surgeons rated the AI-generated summaries above the prespecified acceptability threshold across all domains (mean scores ≥3.0), specifically praising their structure and conciseness. CONCLUSIONS By integrating 3 specialized modules, the proposed framework offers a practical, locally deployable solution for clinician-in-the-loop BHC draft generation under privacy and resource constraints, and holds promise for improving documentation efficiency and enhancing information continuity during care transitions.