Zilei Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zilei Wang has a total of 152 co-authors. Here are the top co-authors by number of shared publications:
- Yixin Zhang
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, Anhui, China
8 shared publications Yixin Zhang
University of Science and Technology of China Hefei China
7 shared publicationsJunjie Li
Beijing University of Posts and Telecommunications
6 shared publications- Tieniu Tan
Institute of Automation, Chinese Academy of Sciences
6 shared publications - Zhilin Li
University of Science and Technology of China
6 shared publications - Jian Yang
University of Science and Technology of China
5 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2778-2785
Ray-Match: Ray-aware One-to-many Training Strategy for Multi-View 3D Object Detection
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2778-2785
Open AccessRay-Match: Ray-aware One-to-many Training Strategy for Multi-View 3D Object Detection
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 11700-11708
ReliaPose: Closing the Loop of Reliability-Aware Temporal Fusion for Video Pose Estimation
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 11700-11708
Open AccessReliaPose: Closing the Loop of Reliability-Aware Temporal Fusion for Video Pose Estimation
- 13 September 2026
- Published by Springer Nature
- p. 402-421
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-task Merging
- 13 September 2026
- Published by Springer Nature
- p. 402-421
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-task Merging
- 13 September 2026
- Published by Springer Nature
- p. 455-472
On the Vulnerability of Parameter-Level Defenses to Model Merging
- 13 September 2026
- Published by Springer Nature
- p. 455-472
On the Vulnerability of Parameter-Level Defenses to Model Merging
- Open Access
Predict the Retrieval! Test time adaptation for Retrieval Augmented Generation
Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for enhancing large language models' question-answering capabilities through the integration of external knowledge. However, when adapting RAG systems to specialized domains, challenges arise from distribution shifts, resulting in suboptimal generalization performance. In this work, we propose TTARAG, a test-time adaptation method that dynamically updates the language model's parameters during inference to improve RAG system performance in specialized domains. Our method introduces a simple yet effective approach where the model learns to predict retrieved content, enabling automatic parameter adjustment to the target domain. Through extensive experiments across six specialized domains, we demonstrate that TTARAG achieves substantial performance improvements over baseline RAG systems. Code available at https://github.com/sunxin000/TTARAG.
Open AccessPredict the Retrieval! Test time adaptation for Retrieval Augmented Generation
Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for enhancing large language models' question-answering capabilities through the integration of external knowledge. However, when adapting RAG systems to specialized domains, challenges arise from distribution shifts, resulting in suboptimal generalization performance. In this work, we propose TTARAG, a test-time adaptation method that dynamically updates the language model's parameters during inference to improve RAG system performance in specialized domains. Our method introduces a simple yet effective approach where the model learns to predict retrieved content, enabling automatic parameter adjustment to the target domain. Through extensive experiments across six specialized domains, we demonstrate that TTARAG achieves substantial performance improvements over baseline RAG systems. Code available at https://github.com/sunxin000/TTARAG.
- 22 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36  (2) , 2304-2317
Improving Zero-Shot Generalization for CLIP With Prompt Ensemble Self-Distillation
Prompt tuning has emerged as an effective alternative for adapting pre-trained Vision-Language Models (VLMs) to various downstream tasks. In our experiments utilizing prompt tuning methods, we observed that modifying the prompt initialization led to inconsistencies in the model’s predictions, particularly with pronounced variability on specific datasets. Motivated by this observation, we examine the predictive performance of two ensemble methods: prompt fusion and logits fusion. Experimental results indicate that logits fusion results in considerable performance improvements, while prompt fusion does not yield any enhancements. However, a significant downside of logits fusion is the enormous rise in inference time. To investigate a practical approach for integrating knowledge derived from multiple prompts without incurring additional inference costs, we propose a straightforward Prompt Ensemble self-Distillation (PED) framework that considerably improves the generalization capacity of prompt tuning. Specifically, we initialize multiple groups of prompts, and during the training process, we integrate the prediction outputs from each group to facilitate the learning of the fused prompts. The proposed self-distillation approach offers dual benefits: enhancing the performance of both the fused prompts and the fused logits. We utilize fused prompts for prediction during the inference process, thereby achieving performance that is comparable to that of fused logits without incurring additional inference time. We evaluate the effectiveness of our methodology across four distinct tasks. Our PED consistently demonstrates superior performance in all assessments when contrasted with numerous state-of-the-art methods. Moreover, our method can be seamlessly integrated into existing prompt learning approaches and consistently improves their performance. Our code is publicly available at https://github.com/vim-wei/PED.
- 22 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36  (2) , 2304-2317
Improving Zero-Shot Generalization for CLIP With Prompt Ensemble Self-Distillation
Prompt tuning has emerged as an effective alternative for adapting pre-trained Vision-Language Models (VLMs) to various downstream tasks. In our experiments utilizing prompt tuning methods, we observed that modifying the prompt initialization led to inconsistencies in the model’s predictions, particularly with pronounced variability on specific datasets. Motivated by this observation, we examine the predictive performance of two ensemble methods: prompt fusion and logits fusion. Experimental results indicate that logits fusion results in considerable performance improvements, while prompt fusion does not yield any enhancements. However, a significant downside of logits fusion is the enormous rise in inference time. To investigate a practical approach for integrating knowledge derived from multiple prompts without incurring additional inference costs, we propose a straightforward Prompt Ensemble self-Distillation (PED) framework that considerably improves the generalization capacity of prompt tuning. Specifically, we initialize multiple groups of prompts, and during the training process, we integrate the prediction outputs from each group to facilitate the learning of the fused prompts. The proposed self-distillation approach offers dual benefits: enhancing the performance of both the fused prompts and the fused logits. We utilize fused prompts for prediction during the inference process, thereby achieving performance that is comparable to that of fused logits without incurring additional inference time. We evaluate the effectiveness of our methodology across four distinct tasks. Our PED consistently demonstrates superior performance in all assessments when contrasted with numerous state-of-the-art methods. Moreover, our method can be seamlessly integrated into existing prompt learning approaches and consistently improves their performance. Our code is publicly available at https://github.com/vim-wei/PED.
- Open Access
Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation
Universal Domain Adaptation (UniDA) focuses on transferring source domain knowledge to the target domain under both domain shift and unknown category shift. Its main challenge lies in identifying common class samples and aligning them. Current methods typically obtain target domain semantics centers from an unconstrained continuous image representation space. Due to domain shift and the unknown number of clusters, these centers often result in complex and less robust alignment algorithm. In this paper, based on vision-language models, we search for semantic centers in a semantically meaningful and discrete text representation space. The constrained space ensures almost no domain bias and appropriate semantic granularity for these centers, enabling a simple and robust adaptation algorithm. Specifically, we propose TArget Semantics Clustering (TASC) via Text Representations, which leverages information maximization as a unified objective and involves two stages. First, with the frozen encoders, a greedy search-based framework is used to search for an optimal set of text embeddings to represent target semantics. Second, with the search results fixed, encoders are refined based on gradient descent, simultaneously achieving robust domain alignment and private class clustering. Additionally, we propose Universal Maximum Similarity (UniMS), a scoring function tailored for detecting open-set samples in UniDA. Experimentally, we evaluate the universality of UniDA algorithms under four category shift scenarios. Extensive experiments on four benchmarks demonstrate the effectiveness and robustness of our method, which has achieved state-of-the-art performance.
Open AccessTarget Semantics Clustering via Text Representations for Robust Universal Domain Adaptation
Universal Domain Adaptation (UniDA) focuses on transferring source domain knowledge to the target domain under both domain shift and unknown category shift. Its main challenge lies in identifying common class samples and aligning them. Current methods typically obtain target domain semantics centers from an unconstrained continuous image representation space. Due to domain shift and the unknown number of clusters, these centers often result in complex and less robust alignment algorithm. In this paper, based on vision-language models, we search for semantic centers in a semantically meaningful and discrete text representation space. The constrained space ensures almost no domain bias and appropriate semantic granularity for these centers, enabling a simple and robust adaptation algorithm. Specifically, we propose TArget Semantics Clustering (TASC) via Text Representations, which leverages information maximization as a unified objective and involves two stages. First, with the frozen encoders, a greedy search-based framework is used to search for an optimal set of text embeddings to represent target semantics. Second, with the search results fixed, encoders are refined based on gradient descent, simultaneously achieving robust domain alignment and private class clustering. Additionally, we propose Universal Maximum Similarity (UniMS), a scoring function tailored for detecting open-set samples in UniDA. Experimentally, we evaluate the universality of UniDA algorithms under four category shift scenarios. Extensive experiments on four benchmarks demonstrate the effectiveness and robustness of our method, which has achieved state-of-the-art performance.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39  (16) , 17132-17140
Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation
Universal Domain Adaptation (UniDA) focuses on transferring source domain knowledge to the target domain under both domain shift and unknown category shift. Its main challenge lies in identifying common class samples and aligning them. Current methods typically obtain target domain semantics centers from an unconstrained continuous image representation space. Due to domain shift and the unknown number of clusters, these centers often result in complex and less robust alignment algorithm. In this paper, based on vision-language models, we search for semantic centers in a semantically meaningful and discrete text representation space. The constrained space ensures almost no domain bias and appropriate semantic granularity for these centers, enabling a simple and robust adaptation algorithm. Specifically, we propose TArget Semantics Clustering (TASC) via Text Representations, which leverages information maximization as a unified objective and involves two stages. First, with the frozen encoders, a greedy search-based framework is used to search for an optimal set of text embeddings to represent target semantics. Second, with the search results fixed, encoders are refined based on gradient descent, simultaneously achieving robust domain alignment and private class clustering. Additionally, we propose Universal Maximum Similarity (UniMS), a scoring function tailored for detecting open-set samples in UniDA. Experimentally, we evaluate the universality of UniDA algorithms under four category shift scenarios. Extensive experiments on four benchmarks demonstrate the effectiveness and robustness of our method, which has achieved state-of-the-art performance.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39  (16) , 17132-17140
Open AccessTarget Semantics Clustering via Text Representations for Robust Universal Domain Adaptation
Universal Domain Adaptation (UniDA) focuses on transferring source domain knowledge to the target domain under both domain shift and unknown category shift. Its main challenge lies in identifying common class samples and aligning them. Current methods typically obtain target domain semantics centers from an unconstrained continuous image representation space. Due to domain shift and the unknown number of clusters, these centers often result in complex and less robust alignment algorithm. In this paper, based on vision-language models, we search for semantic centers in a semantically meaningful and discrete text representation space. The constrained space ensures almost no domain bias and appropriate semantic granularity for these centers, enabling a simple and robust adaptation algorithm. Specifically, we propose TArget Semantics Clustering (TASC) via Text Representations, which leverages information maximization as a unified objective and involves two stages. First, with the frozen encoders, a greedy search-based framework is used to search for an optimal set of text embeddings to represent target semantics. Second, with the search results fixed, encoders are refined based on gradient descent, simultaneously achieving robust domain alignment and private class clustering. Additionally, we propose Universal Maximum Similarity (UniMS), a scoring function tailored for detecting open-set samples in UniDA. Experimentally, we evaluate the universality of UniDA algorithms under four category shift scenarios. Extensive experiments on four benchmarks demonstrate the effectiveness and robustness of our method, which has achieved state-of-the-art performance.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39  (8) , 8015-8023
GCD: Advancing Vision-Language Models for Incremental Object Detection via Global Alignment and Correspondence Distillation
Incremental object detection (IOD) is a challenging task that requires detection models to continuously learn from newly arriving data. This work focuses on incremental learning for vision-language detectors (VLDs), an under explored domain. Existing research typically adopts a local alignment paradigm to avoid label conflicts, where different tasks are learned separately without interaction. However, we reveal that this practice fails to effectively preserve the semantic structure. Specifically, aligned relationships between objects and texts would collapse when handling novel categories, ultimately leading to catastrophic forgetting. Though knowledge distillation (KD) is a common approach for tackling this, traditional KD performs poorly when directly applied to VLDs, as for different phases, a natural knowledge gap exists in both encoding and decoding processes. To address above issues, we propose a novel method called Global alignment and Correspondence Distillation (GCD). Differently, we first integrate knowledge across phases within the same embedding space to construct global semantic structure. We then enable effective knowledge distillation in VLDs through a semantic correspondence mechanism, ensuring consistent proposal generation and decoding. On the top of that, we distill teacher model’s informative predictions and topological relationships to maintain stable local semantic structure. Extensive experiments on COCO 2017 demonstrate that our method significantly outperforms existing approaches, achieving new state-of-the-art in various IOD scenarios.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39  (8) , 8015-8023
Open AccessGCD: Advancing Vision-Language Models for Incremental Object Detection via Global Alignment and Correspondence Distillation
Incremental object detection (IOD) is a challenging task that requires detection models to continuously learn from newly arriving data. This work focuses on incremental learning for vision-language detectors (VLDs), an under explored domain. Existing research typically adopts a local alignment paradigm to avoid label conflicts, where different tasks are learned separately without interaction. However, we reveal that this practice fails to effectively preserve the semantic structure. Specifically, aligned relationships between objects and texts would collapse when handling novel categories, ultimately leading to catastrophic forgetting. Though knowledge distillation (KD) is a common approach for tackling this, traditional KD performs poorly when directly applied to VLDs, as for different phases, a natural knowledge gap exists in both encoding and decoding processes. To address above issues, we propose a novel method called Global alignment and Correspondence Distillation (GCD). Differently, we first integrate knowledge across phases within the same embedding space to construct global semantic structure. We then enable effective knowledge distillation in VLDs through a semantic correspondence mechanism, ensuring consistent proposal generation and decoding. On the top of that, we distill teacher model’s informative predictions and topological relationships to maintain stable local semantic structure. Extensive experiments on COCO 2017 demonstrate that our method significantly outperforms existing approaches, achieving new state-of-the-art in various IOD scenarios.