Zuchao Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zuchao Li has a total of 117 co-authors. Here are the top co-authors by number of shared publications:
- Hai Zhao
Shanghai Jiao Tong University
42 shared publications Hai Zhao
Department of Computer Science and Engineering, Shanghai Jiao Tong University
13 shared publications- Lefei Zhang
School of Computer Science, Wuhan University, Wuhan, China
8 shared publications - Zhuosheng Zhang
Shanghai Jiao Tong University, Shanghai, China
8 shared publications - Ping Wang
School of Information Management, Wuhan University, Wuhan, China
7 shared publications - Rui Wang
Shanghai Jiao Tong University, Shanghai, China
7 shared publications
Latest Publications
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1775-1786
Chain-of-Cognition Through Multi-Agent Collaboration Against Cross-Modal Cognitive Interference
Multi-modal Large Language Models (MLLMs) leverage Chain-of-Thought (CoT) instruction to transfer the reasoning capabilities of Large Language Models (LLMs) into multi-modal contexts. Prior research has primarily focused on aligning multi-modal features with textual representations, while overlooking the fundamental distinction that multi-modal information perception and information-grounded reasoning necessitate distinct cognitive modes. This limitation induces Cross-Modal Cognitive Interference (CMCI) in MLLMs, characterized by mutual interference between visual perception and reasoning capabilities during multi-modal CoT. To systematically analyse this phenomenon, we develop the CMCI Diagnostic Benchmark, which reveals prevalent mutual interference between perception and reasoning capabilities across advanced MLLMs. Furthermore, we propose a multi-agent dual-system Chain-of-Cognition (CoC) framework, called MADC, to mitigate CMCI in visual question answering. Unlike the traditional CoT methodology, the CoC is explicitly structured into four distinct stages: perception, assessment, reasoning, and decision-making. This structured decomposition enables MADC to better address CMCI, making it more effective for complex multi-modal reasoning tasks. Our empirical evaluation across multiple benchmarks shows that MADC significantly surpasses conventional MLLMs and demonstrates strong compatibility with various multi-modal and uni-modal foundation models. Experiments demonstrate that the MADC framework outperforms end-to-end multi-modal slow-thinking models without the need for fine-tuning, validating the effectiveness of our approach. Additionally, experiments on the video question answering benchmark further validate the generalizability of the MADC framework across multiple modalities.
- 19 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 34, 1775-1786
Chain-of-Cognition Through Multi-Agent Collaboration Against Cross-Modal Cognitive Interference
Multi-modal Large Language Models (MLLMs) leverage Chain-of-Thought (CoT) instruction to transfer the reasoning capabilities of Large Language Models (LLMs) into multi-modal contexts. Prior research has primarily focused on aligning multi-modal features with textual representations, while overlooking the fundamental distinction that multi-modal information perception and information-grounded reasoning necessitate distinct cognitive modes. This limitation induces Cross-Modal Cognitive Interference (CMCI) in MLLMs, characterized by mutual interference between visual perception and reasoning capabilities during multi-modal CoT. To systematically analyse this phenomenon, we develop the CMCI Diagnostic Benchmark, which reveals prevalent mutual interference between perception and reasoning capabilities across advanced MLLMs. Furthermore, we propose a multi-agent dual-system Chain-of-Cognition (CoC) framework, called MADC, to mitigate CMCI in visual question answering. Unlike the traditional CoT methodology, the CoC is explicitly structured into four distinct stages: perception, assessment, reasoning, and decision-making. This structured decomposition enables MADC to better address CMCI, making it more effective for complex multi-modal reasoning tasks. Our empirical evaluation across multiple benchmarks shows that MADC significantly surpasses conventional MLLMs and demonstrates strong compatibility with various multi-modal and uni-modal foundation models. Experiments demonstrate that the MADC framework outperforms end-to-end multi-modal slow-thinking models without the need for fine-tuning, validating the effectiveness of our approach. Additionally, experiments on the video question answering benchmark further validate the generalizability of the MADC framework across multiple modalities.
- 13 February 2026
- Published by Elsevier in Neural Networks
- Vol. 198, 108720
Generalized and group spherical linear interpolation for token-level context compression
- 13 February 2026
- Published by Elsevier in Neural Networks
- Vol. 198, 108720
Generalized and group spherical linear interpolation for token-level context compression
- 9 November 2025
- Published by Springer Nature
- p. 214-228
DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Models
- 9 November 2025
- Published by Springer Nature
- p. 214-228
DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Models
- Open Access
OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
Recent advances in multimodal large language models (LLMs) have led to significant progress in understanding, generation, and retrieval tasks. However, current solutions often treat these tasks in isolation or require training LLMs from scratch, resulting in high computational costs and limited generalization across modalities. In this work, we present OmniBridge, a unified and modular multimodal framework that supports vision-language understanding, generation, and retrieval within a unified architecture. OmniBridge adopts a language-centric design that reuses pretrained LLMs and introduces a lightweight bidirectional latent alignment module. To address the challenge of task interference, we propose a two-stage decoupled training strategy: supervised fine-tuning and latent space alignment for aligning LLM behavior with multimodal reasoning, and semantic-guided diffusion training to align cross-modal latent spaces via learnable query embeddings. Extensive experiments across a wide range of benchmarks demonstrate that OmniBridge achieves competitive or state-of-the-art performance in all three tasks. Moreover, our results highlight the effectiveness of latent space alignment for unifying multimodal modeling under a shared representation space. Code and models are released at https://github.com/xiao-xt/OmniBridge.
Open AccessOmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignment
Recent advances in multimodal large language models (LLMs) have led to significant progress in understanding, generation, and retrieval tasks. However, current solutions often treat these tasks in isolation or require training LLMs from scratch, resulting in high computational costs and limited generalization across modalities. In this work, we present OmniBridge, a unified and modular multimodal framework that supports vision-language understanding, generation, and retrieval within a unified architecture. OmniBridge adopts a language-centric design that reuses pretrained LLMs and introduces a lightweight bidirectional latent alignment module. To address the challenge of task interference, we propose a two-stage decoupled training strategy: supervised fine-tuning and latent space alignment for aligning LLM behavior with multimodal reasoning, and semantic-guided diffusion training to align cross-modal latent spaces via learnable query embeddings. Extensive experiments across a wide range of benchmarks demonstrate that OmniBridge achieves competitive or state-of-the-art performance in all three tasks. Moreover, our results highlight the effectiveness of latent space alignment for unifying multimodal modeling under a shared representation space. Code and models are released at https://github.com/xiao-xt/OmniBridge.
- Top Cited Papers
- 22 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34  (10577149) , 7378-7390
Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning
Recently, while significant progress has been made in remote sensing image change captioning, existing methods fail to filter out areas unrelated to actual changes, making models susceptible to irrelevant features. In this article, we propose a novel multimodal model for remote sensing image change captioning, guided by Key Change Features and Instruction-tuned (KCFI). This model aims to fully leverage the intrinsic knowledge of large language models through visual instructions and enhance the effectiveness and accuracy of change features using pixel-level change detection tasks. Specifically, KCFI includes a ViTs encoder for extracting bi-temporal remote sensing image features, a key feature perceiver for identifying critical change areas, a pixel-level change detection decoder to constrain key change features, and an instruction-tuned decoder based on a large language model. Moreover, to ensure that change captioning and change detection tasks are jointly optimized, we employ a dynamic weight-averaging strategy to balance the losses between the two tasks. We also explore various feature combinations for visual fine-tuning instructions and demonstrate that using only key change features to guide the large language model is the optimal choice. To validate the effectiveness of our approach, we compare it against several state-of-the-art change captioning methods on the LEVIR-CC dataset, achieving the best performance. Our code will be available at https://github.com/yangcong356/KCFI.git.
- 22 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34  (10577149) , 7378-7390
Top Cited PapersEnhancing Perception of Key Changes in Remote Sensing Image Change Captioning
Recently, while significant progress has been made in remote sensing image change captioning, existing methods fail to filter out areas unrelated to actual changes, making models susceptible to irrelevant features. In this article, we propose a novel multimodal model for remote sensing image change captioning, guided by Key Change Features and Instruction-tuned (KCFI). This model aims to fully leverage the intrinsic knowledge of large language models through visual instructions and enhance the effectiveness and accuracy of change features using pixel-level change detection tasks. Specifically, KCFI includes a ViTs encoder for extracting bi-temporal remote sensing image features, a key feature perceiver for identifying critical change areas, a pixel-level change detection decoder to constrain key change features, and an instruction-tuned decoder based on a large language model. Moreover, to ensure that change captioning and change detection tasks are jointly optimized, we employ a dynamic weight-averaging strategy to balance the losses between the two tasks. We also explore various feature combinations for visual fine-tuning instructions and demonstrate that using only key change features to guide the large language model is the optimal choice. To validate the effectiveness of our approach, we compare it against several state-of-the-art change captioning methods on the LEVIR-CC dataset, achieving the best performance. Our code will be available at https://github.com/yangcong356/KCFI.git.
- Open Access
IAM: Efficient Inference through Attention Mapping between Different-scale LLMs
LLMs encounter significant challenges in resource consumption nowadays, especially with long contexts. Despite extensive efforts dedicate to enhancing inference efficiency, these methods primarily exploit internal sparsity within the models, without leveraging external information for optimization. We identify the high similarity of attention matrices across different-scale LLMs, which offers a novel perspective for optimization. We first conduct a comprehensive analysis of how to measure similarity, how to select mapping Layers and whether mapping is consistency. Based on these insights, we introduce the IAM framework, which achieves dual benefits of accelerated attention computation and reduced KV cache usage by performing attention mapping between small and large LLMs. Our experimental results demonstrate that IAM can accelerate prefill by 15% and reduce KV cache usage by 22.1% without appreciably sacrificing performance. Experiments on different series of models show the generalizability of IAM. Importantly, it is also orthogonal to many existing KV cache optimization methods, making it a versatile addition to the current toolkit for enhancing LLM efficiency.
Open AccessIAM: Efficient Inference through Attention Mapping between Different-scale LLMs
LLMs encounter significant challenges in resource consumption nowadays, especially with long contexts. Despite extensive efforts dedicate to enhancing inference efficiency, these methods primarily exploit internal sparsity within the models, without leveraging external information for optimization. We identify the high similarity of attention matrices across different-scale LLMs, which offers a novel perspective for optimization. We first conduct a comprehensive analysis of how to measure similarity, how to select mapping Layers and whether mapping is consistency. Based on these insights, we introduce the IAM framework, which achieves dual benefits of accelerated attention computation and reduced KV cache usage by performing attention mapping between small and large LLMs. Our experimental results demonstrate that IAM can accelerate prefill by 15% and reduce KV cache usage by 22.1% without appreciably sacrificing performance. Experiments on different series of models show the generalizability of IAM. Importantly, it is also orthogonal to many existing KV cache optimization methods, making it a versatile addition to the current toolkit for enhancing LLM efficiency.
- Open Access
DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
Task-agnostic prompt compression leverages the redundancy in natural language to reduce computational overhead and enhance information density within prompts, especially in long-context scenarios. Existing methods predominantly rely on information entropy as the metric to compress lexical units, aiming to achieve minimal information loss. However, these approaches overlook two critical aspects: (i) the importance of attention-critical tokens at the algorithmic level, and (ii) shifts in information entropy during the compression process. Motivated by these challenges, we propose a dynamic attention-aware approach for task-agnostic prompt compression (DAC). This approach effectively integrates entropy and attention information, dynamically sensing entropy shifts during compression to achieve fine-grained prompt compression. Extensive experiments across various domains, including LongBench, GSM8K, and BBH, show that DAC consistently yields robust and substantial improvements across a diverse range of tasks and LLMs, offering compelling evidence of its efficacy.
Open AccessDAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression
Task-agnostic prompt compression leverages the redundancy in natural language to reduce computational overhead and enhance information density within prompts, especially in long-context scenarios. Existing methods predominantly rely on information entropy as the metric to compress lexical units, aiming to achieve minimal information loss. However, these approaches overlook two critical aspects: (i) the importance of attention-critical tokens at the algorithmic level, and (ii) shifts in information entropy during the compression process. Motivated by these challenges, we propose a dynamic attention-aware approach for task-agnostic prompt compression (DAC). This approach effectively integrates entropy and attention information, dynamically sensing entropy shifts during compression to achieve fine-grained prompt compression. Extensive experiments across various domains, including LongBench, GSM8K, and BBH, show that DAC consistently yields robust and substantial improvements across a diverse range of tasks and LLMs, offering compelling evidence of its efficacy.
- Open Access
- 8 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 87015-87025
Document-Level Neural Machine Translation With Document Embeddings
Standard neural machine translation (NMT) assumes that document-level context information is irrespective. Most existing document-level NMT methods are satisfied with a smattering sense of shallow document-level information, such as using a few context sentences surrounding the source sentence as document-level information. Our work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings, which can sufficiently model deeper and richer document-level context. The proposed document-level NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end. We compress the entire document text with explicit boundaries into a token-size global static document embedding, and the neighboring sentences as a token-size local dynamic document embedding and concatenate with the source tokens. Experiments reveal that the proposed method significantly improves the translation performance over strong baselines and other related studies.
- 8 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 87015-87025
Open AccessDocument-Level Neural Machine Translation With Document Embeddings
Standard neural machine translation (NMT) assumes that document-level context information is irrespective. Most existing document-level NMT methods are satisfied with a smattering sense of shallow document-level information, such as using a few context sentences surrounding the source sentence as document-level information. Our work focuses on exploiting detailed document-level context in terms of multiple forms of document embeddings, which can sufficiently model deeper and richer document-level context. The proposed document-level NMT is implemented to enhance the Transformer baseline by introducing both global and local document-level clues on the source end. We compress the entire document text with explicit boundaries into a token-size global static document embedding, and the neighboring sentences as a token-size local dynamic document embedding and concatenate with the source tokens. Experiments reveal that the proposed method significantly improves the translation performance over strong baselines and other related studies.
- Open Access
Model Hemorrhage and the Robustness Limits of Large Language Models
Large language models (LLMs) demonstrate strong performance across natural language processing tasks, yet undergo significant performance degradation when modified for deployment through quantization, pruning, or decoding strategy adjustments. We define this phenomenon as model hemorrhage - performance decline caused by parameter alterations and architectural changes. Through systematic analysis of various LLM frameworks, we identify key vulnerability patterns: layer expansion frequently disrupts attention mechanisms, compression techniques induce information loss cascades, and decoding adjustments amplify prediction divergences. Our investigation reveals transformer architectures exhibit inherent robustness thresholds that determine hemorrhage severity across modification types. We propose three mitigation strategies: gradient-aware pruning preserves critical weight pathways, dynamic quantization scaling maintains activation integrity, and decoding calibration aligns generation trajectories with original model distributions. This work establishes foundational metrics for evaluating model stability during adaptation, providing practical guidelines for maintaining performance while enabling efficient LLM deployment. Our findings advance understanding of neural network resilience under architectural transformations, particularly for large-scale language models.
Open AccessModel Hemorrhage and the Robustness Limits of Large Language Models
Large language models (LLMs) demonstrate strong performance across natural language processing tasks, yet undergo significant performance degradation when modified for deployment through quantization, pruning, or decoding strategy adjustments. We define this phenomenon as model hemorrhage - performance decline caused by parameter alterations and architectural changes. Through systematic analysis of various LLM frameworks, we identify key vulnerability patterns: layer expansion frequently disrupts attention mechanisms, compression techniques induce information loss cascades, and decoding adjustments amplify prediction divergences. Our investigation reveals transformer architectures exhibit inherent robustness thresholds that determine hemorrhage severity across modification types. We propose three mitigation strategies: gradient-aware pruning preserves critical weight pathways, dynamic quantization scaling maintains activation integrity, and decoding calibration aligns generation trajectories with original model distributions. This work establishes foundational metrics for evaluating model stability during adaptation, providing practical guidelines for maintaining performance while enabling efficient LLM deployment. Our findings advance understanding of neural network resilience under architectural transformations, particularly for large-scale language models.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 23968-23982
Can Large Language Models Be Good Language Teachers?
LiQing Xu, Qiwei Li, Tianshuo Peng, Zuchao Li, Hai Zhao, Ping Wang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 23968-23982
Open AccessCan Large Language Models Be Good Language Teachers?
LiQing Xu, Qiwei Li, Tianshuo Peng, Zuchao Li, Hai Zhao, Ping Wang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.