Shide Zhou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shide Zhou has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Kailong Wang
National University of Singapore
5 shared publications - Haoyu Wang
HuaZhong University of Science and Technology, Wuhan, Hubei, China
3 shared publications - Ling Shi
Nanyang technological university, Singapore, Singapore
3 shared publications - Jiaqing Li
Huazhong University of Science and Technology
1 shared publication - Jiaxuan Liang
Huazhong University of Science and Technology, Wuhan, China
1 shared publication - Tianlin Li
Nanyang technological university, Singapore, Singapore
1 shared publication
Latest Publications
- Open Access
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (ISSTA) , 1194-1216
A Temporal Reasoning Benchmarking Framework for LRMs via Difficulty-Controlled and Dynamic Test Generation
Defining the reasoning boundaries and ensuring the reliability of Large Reasoning Models (LRMs) remains a critical challenge. Current benchmarks primarily rely on static datasets susceptible to data contamination or synthetic tasks lacking fine-grained difficulty control. Furthermore, standard outcome-based evaluations often conceal reasoning flaws by neglecting the reasoning process. To address these limitations, we introduce TRACE, a testing framework that models temporal reasoning as constraint satisfaction problems via Allen’s Interval Algebra. This approach enables precise regulation of logical complexity and incorporates a Trace-Based Verification Oracle to validate reasoning faithfulness. Using this framework, we construct TRACEBench, an extensive benchmark comprising 1,200 synthesized test instances across graded difficulty levels. We employ TRACE to evaluate eight widely used LRMs on TRACEBench. The results confirm a strong negative correlation between model performance and our difficulty metric (Pearson’s r ≈ −0.96), validating the effectiveness of our difficulty control mechanism. Moreover, our trace-based analysis exposes significant discrepancies between reasoning validity and final answers, revealing a high spurious guessing rate of approximately 28% in mid-sized models. In addition, we diagnose scale-dependent failure modes, ranging from Degenerative Loops in small models to Reasoning Explosion in advanced architectures. TRACE thus provides a robust, automated platform for benchmarking the true temporal reasoning capabilities of LRMs.
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (ISSTA) , 1194-1216
Open AccessA Temporal Reasoning Benchmarking Framework for LRMs via Difficulty-Controlled and Dynamic Test Generation
Defining the reasoning boundaries and ensuring the reliability of Large Reasoning Models (LRMs) remains a critical challenge. Current benchmarks primarily rely on static datasets susceptible to data contamination or synthetic tasks lacking fine-grained difficulty control. Furthermore, standard outcome-based evaluations often conceal reasoning flaws by neglecting the reasoning process. To address these limitations, we introduce TRACE, a testing framework that models temporal reasoning as constraint satisfaction problems via Allen’s Interval Algebra. This approach enables precise regulation of logical complexity and incorporates a Trace-Based Verification Oracle to validate reasoning faithfulness. Using this framework, we construct TRACEBench, an extensive benchmark comprising 1,200 synthesized test instances across graded difficulty levels. We employ TRACE to evaluate eight widely used LRMs on TRACEBench. The results confirm a strong negative correlation between model performance and our difficulty metric (Pearson’s r ≈ −0.96), validating the effectiveness of our difficulty control mechanism. Moreover, our trace-based analysis exposes significant discrepancies between reasoning validity and final answers, revealing a high spurious guessing rate of approximately 28% in mid-sized models. In addition, we diagnose scale-dependent failure modes, ranging from Degenerative Loops in small models to Reasoning Explosion in advanced architectures. TRACE thus provides a robust, automated platform for benchmarking the true temporal reasoning capabilities of LRMs.
- 4 August 2026
- Published by Springer Nature
- p. 575-589
When Safe Models Merge Into Danger: Exploiting Latent Vulnerabilities in LLM Fusion
- 4 August 2026
- Published by Springer Nature
- p. 575-589
When Safe Models Merge Into Danger: Exploiting Latent Vulnerabilities in LLM Fusion
- 13 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 21 (15566013) , 4074-4089
Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics
The widespread adoption of Large Language Models (LLMs) in critical applications has introduced severe reliability and security risks, as LLMs remain vulnerable to notorious threats such as hallucinations, jailbreak attacks, and backdoor exploits. These vulnerabilities have been weaponized by malicious actors, leading to unauthorized access, widespread misinformation, and compromised LLM-embedded system integrity. In this work, we introduce a novel approach to detecting abnormal behaviors in LLMs via hidden state forensics. By systematically inspecting layer-specific activation patterns, we develop a general framework that can efficiently identify a range of security threats in real-time without imposing prohibitive computational costs. Extensive experiments indicate detection accuracies exceeding 95% and consistently robust performance across multiple models in most scenarios, while preserving the ability to detect novel attacks effectively. Furthermore, the computational overhead remains minimal, with detector inference taking merely fractions of a second. The significance of this work lies in proposing a promising strategy to reinforce the security of LLM-integrated systems, paving the way for safer and more reliable deployment in high-stakes domains. By enabling real-time detection that can also support the mitigation of abnormal behaviors, it represents a meaningful step toward ensuring the trustworthiness of AI systems amid rising security challenges.
- 13 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 21 (15566013) , 4074-4089
Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics
The widespread adoption of Large Language Models (LLMs) in critical applications has introduced severe reliability and security risks, as LLMs remain vulnerable to notorious threats such as hallucinations, jailbreak attacks, and backdoor exploits. These vulnerabilities have been weaponized by malicious actors, leading to unauthorized access, widespread misinformation, and compromised LLM-embedded system integrity. In this work, we introduce a novel approach to detecting abnormal behaviors in LLMs via hidden state forensics. By systematically inspecting layer-specific activation patterns, we develop a general framework that can efficiently identify a range of security threats in real-time without imposing prohibitive computational costs. Extensive experiments indicate detection accuracies exceeding 95% and consistently robust performance across multiple models in most scenarios, while preserving the ability to detect novel attacks effectively. Furthermore, the computational overhead remains minimal, with detector inference taking merely fractions of a second. The significance of this work lies in proposing a promising strategy to reinforce the security of LLM-integrated systems, paving the way for safer and more reliable deployment in high-stakes domains. By enabling real-time detection that can also support the mitigation of abnormal behaviors, it represents a meaningful step toward ensuring the trustworthiness of AI systems amid rising security challenges.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 228-232
OmniBench-RAG: A Multi-Domain Evaluation Platform for Retrieval-Augmented Generation Tools
While Retrieval Augmented Generation (RAG) is widely adopted to enhance LLMs, evaluating its true performance benefits in a reproducible and interpretable way remains challenging. Existing methods often fall short: they lack domain coverage, employ coarse metrics that miss sub-document precision, fail to capture computational trade-offs, and suffer from data leakage in some evaluation datasets (leading to unfair assessment results). Most critically, they provide no standardized framework for comparing RAG effectiveness across different models and domains. We introduce OmniBench-RAG, a novel automated platform for multi-domain evaluation of RAG systems. It supports user modification of multiple parameters to investigate RAG performance correlations with various factors, including multi-domain evaluation datasets, configurable reasoning settings, curated or user-provided RAG corpora, and flexible retrieval parameters such as Top-K. The platform quantifies performance using two standardized metrics: Improvements (accuracy gains) and Transformation (efficiency differences between pre-RAG and post-RAG models), enabling reproducible comparisons across models and tasks. It also features dynamic test generation, modular evaluation pipelines, and automated knowledge base construction. The evaluation reveals that RAG improvements are not guaranteed; rather, they depend strongly on factors such as corpus–dataset relevance and retrieval configurations, which the platform systematically exposes. The code and datasets are available at: https://github.com/Garnett-Liang/OmniBench-RAG.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 228-232
Open AccessOmniBench-RAG: A Multi-Domain Evaluation Platform for Retrieval-Augmented Generation Tools
While Retrieval Augmented Generation (RAG) is widely adopted to enhance LLMs, evaluating its true performance benefits in a reproducible and interpretable way remains challenging. Existing methods often fall short: they lack domain coverage, employ coarse metrics that miss sub-document precision, fail to capture computational trade-offs, and suffer from data leakage in some evaluation datasets (leading to unfair assessment results). Most critically, they provide no standardized framework for comparing RAG effectiveness across different models and domains. We introduce OmniBench-RAG, a novel automated platform for multi-domain evaluation of RAG systems. It supports user modification of multiple parameters to investigate RAG performance correlations with various factors, including multi-domain evaluation datasets, configurable reasoning settings, curated or user-provided RAG corpora, and flexible retrieval parameters such as Top-K. The platform quantifies performance using two standardized metrics: Improvements (accuracy gains) and Transformation (efficiency differences between pre-RAG and post-RAG models), enabling reproducible comparisons across models and tasks. It also features dynamic test generation, modular evaluation pipelines, and automated knowledge base construction. The evaluation reveals that RAG improvements are not guaranteed; rather, they depend strongly on factors such as corpus–dataset relevance and retrieval configurations, which the platform systematically exposes. The code and datasets are available at: https://github.com/Garnett-Liang/OmniBench-RAG.
- Open Access
- 11 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (1) , 1-27
NeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-Level Semantic Slicing
Deep Neural Networks (DNNs), extensively applied across diverse disciplines, are characterized by their integrated and monolithic architectures, setting them apart from conventional software systems. This architectural difference introduces particular challenges to maintenance tasks, such as model restructure (e.g., model compression), re-adaptation (e.g., fitting new samples), and incremental development (e.g., continual knowledge accumulation). Prior research addresses these challenges by identifying task-critical neuron layers and dividing neural networks into semantically similar sequential modules. However, such layer-level approaches fail to precisely identify and manipulate neuron-level semantic components, restricting their applicability to finer-grained model maintenance tasks. In this work, we implement NeuSemSlice, a novel framework that introduces the semantic slicing technique to effectively identify critical neuron-level semantic components in DNN models for semantic-aware model maintenance tasks. Specifically, semantic slicing identifies, categorizes, and merges critical neurons across different categories and layers according to their semantic similarity, enabling their flexibility and effectiveness in the subsequent tasks. For semantic-aware model maintenance tasks, we provide a series of novel strategies based on semantic slicing to enhance NeuSemSlice. They include semantic components (i.e., critical neurons) preservation for model restructure, critical neuron tuning for model re-adaptation, and non-critical neuron training for model incremental development. A thorough evaluation has demonstrated that NeuSemSlice significantly outperforms baselines in all three tasks.
- 11 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (1) , 1-27
Open AccessNeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-Level Semantic Slicing
Deep Neural Networks (DNNs), extensively applied across diverse disciplines, are characterized by their integrated and monolithic architectures, setting them apart from conventional software systems. This architectural difference introduces particular challenges to maintenance tasks, such as model restructure (e.g., model compression), re-adaptation (e.g., fitting new samples), and incremental development (e.g., continual knowledge accumulation). Prior research addresses these challenges by identifying task-critical neuron layers and dividing neural networks into semantically similar sequential modules. However, such layer-level approaches fail to precisely identify and manipulate neuron-level semantic components, restricting their applicability to finer-grained model maintenance tasks. In this work, we implement NeuSemSlice, a novel framework that introduces the semantic slicing technique to effectively identify critical neuron-level semantic components in DNN models for semantic-aware model maintenance tasks. Specifically, semantic slicing identifies, categorizes, and merges critical neurons across different categories and layers according to their semantic similarity, enabling their flexibility and effectiveness in the subsequent tasks. For semantic-aware model maintenance tasks, we provide a series of novel strategies based on semantic slicing to enhance NeuSemSlice. They include semantic components (i.e., critical neurons) preservation for model restructure, critical neuron tuning for model re-adaptation, and non-critical neuron training for model incremental development. A thorough evaluation has demonstrated that NeuSemSlice significantly outperforms baselines in all three tasks.