Wei Ma
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Wei Ma has a total of 79 co-authors. Here are the top co-authors by number of shared publications:
- Yang Liu
Department of Obstetrics and Gynecology, The Second Affiliated Hospital of Kunming Medical University, China
15 shared publications - Qiang Hu
Tianjin University, Tianjin, China
7 shared publications - Xiaofei Xie
Singapore Management University, Singapore, Singapore
7 shared publications - Shangqing Liu
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
5 shared publications - Cen Zhang
Nanyang technological university, Singapore, Singapore
4 shared publications - Jian Zhang
Nanyang technological university, Singapore, Singapore
4 shared publications
Latest Publications
- Open Access
- 3 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (11) , 1-35
Arash : Token-Efficient LLM-Assisted Crash Root Cause Analysis in Fuzz Driver Generation
Fuzz driver generation (FDG) has improved coverage and automation in modern fuzzing. Yet a persistent challenge remains: many crashes in generated drivers stem from incorrect API usage rather than genuine library defects. Consequently, root cause analysis (RCA) becomes essential for distinguishing true library bugs from driver-induced crashes. However, existing FDG systems lack dedicated RCA capabilities, whereas statistical RCA methods, representative of recent top-tier works (e.g., Aurora, Racing, and OptRCA), rely on simplifying assumptions, such as treating the first observable divergence as the true cause, which often fail to capture the underlying fault semantics in practice. In this article, we propose Arash , an RCA framework for fuzz driver crashes that couples semantic checking with token-efficient code slicing to enable precise and efficient analysis. Arash introduces two complementary components: (i) Fuzz Driver–Oriented Checker , which captures two dominant classes of API misuse in the fuzz driver, namely property- and extent-related crashes , and correlates these violations with crash traces and (ii) Parameter-Guided Code Slicing , which reduces token consumption and contextual noise by preserving only statements relevant to parameters involved in the crash. We implement and evaluate Arash across a diverse corpus of fuzz driver crashes. Arash achieves a recall rate of 85.3%, successfully analyzing cases that all other tools fail to handle, and outperforms the current state-of-the-art by 52.6% in diagnostic accuracy. Moreover, Arash is highly token efficient, achieving an average token reduction of 46.8% relative to a large language model (LLM) baseline that consumes windowed raw code context and 24.5% relative to a retrieval-augmented LLM baseline that retrieves full stack–frame functions. Integrated into OSS-Fuzz-Gen, so far 30 genuine library bugs identified by Arash have been confirmed by developers, including one assigned a CVE (CVE-2024-9143), demonstrating its substantial real-world impact in large-scale fuzzing workflows.
- 3 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (11) , 1-35
Open AccessArash : Token-Efficient LLM-Assisted Crash Root Cause Analysis in Fuzz Driver Generation
Fuzz driver generation (FDG) has improved coverage and automation in modern fuzzing. Yet a persistent challenge remains: many crashes in generated drivers stem from incorrect API usage rather than genuine library defects. Consequently, root cause analysis (RCA) becomes essential for distinguishing true library bugs from driver-induced crashes. However, existing FDG systems lack dedicated RCA capabilities, whereas statistical RCA methods, representative of recent top-tier works (e.g., Aurora, Racing, and OptRCA), rely on simplifying assumptions, such as treating the first observable divergence as the true cause, which often fail to capture the underlying fault semantics in practice. In this article, we propose Arash , an RCA framework for fuzz driver crashes that couples semantic checking with token-efficient code slicing to enable precise and efficient analysis. Arash introduces two complementary components: (i) Fuzz Driver–Oriented Checker , which captures two dominant classes of API misuse in the fuzz driver, namely property- and extent-related crashes , and correlates these violations with crash traces and (ii) Parameter-Guided Code Slicing , which reduces token consumption and contextual noise by preserving only statements relevant to parameters involved in the crash. We implement and evaluate Arash across a diverse corpus of fuzz driver crashes. Arash achieves a recall rate of 85.3%, successfully analyzing cases that all other tools fail to handle, and outperforms the current state-of-the-art by 52.6% in diagnostic accuracy. Moreover, Arash is highly token efficient, achieving an average token reduction of 46.8% relative to a large language model (LLM) baseline that consumes windowed raw code context and 24.5% relative to a retrieval-augmented LLM baseline that retrieves full stack–frame functions. Integrated into OSS-Fuzz-Gen, so far 30 genuine library bugs identified by Arash have been confirmed by developers, including one assigned a CVE (CVE-2024-9143), demonstrating its substantial real-world impact in large-scale fuzzing workflows.
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (11) , 3512-3526
ConfuScope: A Multi-Agent Framework via Hybrid Search and Content-Aware Analysis for Robust Package Confusion Detection
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (11) , 3512-3526
ConfuScope: A Multi-Agent Framework via Hybrid Search and Content-Aware Analysis for Robust Package Confusion Detection
- Open Access
- 26 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
Code analysis is a fundamental problem in Software Engineering (SE), playing a crucial role in tasks such as debugging, performance optimization, and security assessment. Human developers typically approach code analysis through syntax parsing, static semantics inference, and dynamic reasoning. Traditional code analysis tools, while effective, are often limited by language specificity, complex configurations, and lack of cross-language generalization. Recently, large language models (LLMs) have emerged as promising tools for automating various code-related tasks. However, their capabilities for fundamental code analysis remain underexplored. Understanding these capabilities is crucial for assessing and enhancing LLM-based programming tools. We structure our study around three aspects of code analysis aligned with human practices: syntax parsing, static semantics inference, and dynamic reasoning. We evaluate 21 state-of-the-art LLMs across nine representative tasks spanning multiple programming languages (C, Java, Python, Solidity), including Abstract Syntax Tree (AST) generation, Control Flow Graph (CFG) construction, data dependency analysis, taint analysis, and flaky test reasoning. We apply a rigorous three-layer evaluation protocol (combining automated metrics, expert adjudication, and consistency validation) to 3,124 code samples. The protocol achieves high inter-rater reliability (Cohen's \(\kappa=0.844\) – \(0.936\) ) and strong human-machine agreement (Gwet's AC1 = 0.500–0.727, F1 = 0.791–0.882), ensuring methodological rigor. Our results reveal that while best-performing LLMs excel in syntax parsing (AST 90%+, expression matching 84–100%) and show initial promise in static analysis, their dynamic reasoning capabilities remain limited ( \(<\) 70%) with high data-shift sensitivity (F1 scores varying 0–1.0 across projects). This capability hierarchy (strong syntax parsing, moderate static analysis, weak dynamic reasoning) appears consistently across different model families and scales, suggesting fundamental rather than transient limitations. These findings offer new insights into how LLMs can complement traditional code analyzers: LLMs provide cross-language generalization but produce non-deterministic outputs requiring validation, while traditional tools provide deterministic guarantees but require language-specific configuration. We contribute a validated evaluation framework with systematic comparison against traditional analyzers (Tree-sitter, Soot, Joern), task-specific applicability tiers for deployment guidance, and clarify directions for future enhancement of LLM-based code analysis. Our open benchmark is released at https://github.com/mathieu0905/llm_code_analysis.git .- 26 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessExploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
Code analysis is a fundamental problem in Software Engineering (SE), playing a crucial role in tasks such as debugging, performance optimization, and security assessment. Human developers typically approach code analysis through syntax parsing, static semantics inference, and dynamic reasoning. Traditional code analysis tools, while effective, are often limited by language specificity, complex configurations, and lack of cross-language generalization. Recently, large language models (LLMs) have emerged as promising tools for automating various code-related tasks. However, their capabilities for fundamental code analysis remain underexplored. Understanding these capabilities is crucial for assessing and enhancing LLM-based programming tools. We structure our study around three aspects of code analysis aligned with human practices: syntax parsing, static semantics inference, and dynamic reasoning. We evaluate 21 state-of-the-art LLMs across nine representative tasks spanning multiple programming languages (C, Java, Python, Solidity), including Abstract Syntax Tree (AST) generation, Control Flow Graph (CFG) construction, data dependency analysis, taint analysis, and flaky test reasoning. We apply a rigorous three-layer evaluation protocol (combining automated metrics, expert adjudication, and consistency validation) to 3,124 code samples. The protocol achieves high inter-rater reliability (Cohen's \(\kappa=0.844\) – \(0.936\) ) and strong human-machine agreement (Gwet's AC1 = 0.500–0.727, F1 = 0.791–0.882), ensuring methodological rigor. Our results reveal that while best-performing LLMs excel in syntax parsing (AST 90%+, expression matching 84–100%) and show initial promise in static analysis, their dynamic reasoning capabilities remain limited ( \(<\) 70%) with high data-shift sensitivity (F1 scores varying 0–1.0 across projects). This capability hierarchy (strong syntax parsing, moderate static analysis, weak dynamic reasoning) appears consistently across different model families and scales, suggesting fundamental rather than transient limitations. These findings offer new insights into how LLMs can complement traditional code analyzers: LLMs provide cross-language generalization but produce non-deterministic outputs requiring validation, while traditional tools provide deterministic guarantees but require language-specific configuration. We contribute a validated evaluation framework with systematic comparison against traditional analyzers (Tree-sitter, Soot, Joern), task-specific applicability tiers for deployment guidance, and clarify directions for future enhancement of LLM-based code analysis. Our open benchmark is released at https://github.com/mathieu0905/llm_code_analysis.git . - Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1532-1544
Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1532-1544
Open AccessBeyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 3028-3039
MazeBreaker: Multi-Agent Reinforcement Learning for Dynamic Jailbreaking of LLM Security Defenses
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 3028-3039
Open AccessMazeBreaker: Multi-Agent Reinforcement Learning for Dynamic Jailbreaking of LLM Security Defenses
- 2 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (4) , 1549-1567
Towards Secure Program Partitioning for Smart Contracts With LLM’s In-Context Learning
Smart contracts are highly susceptible to manipulation attacks due to the leakage of sensitive information. Addressing manipulation vulnerabilities is particularly challenging because they stem from inherent data confidentiality issues rather than straightforward implementation bugs. To tackle this by preventing sensitive information leakage, we present PARTITIONGPT, the first LLM-driven approach that combines static analysis with the in-context learning capabilities of large language models (LLMs) to partition smart contracts into critical (privileged) and normal codebases, guided by a few annotated sensitive data variables. We evaluated PARTITIONGPT on 18 annotated smart contracts containing 99 sensitive functions. The results demonstrate that PARTITIONGPT successfully generates compilable, and verified partitions, achieving a precision of 80% while reducing more than 26% code compared to functionlevel partitioning approach. Furthermore, we evaluated PARTITIONGPT on nine real-world manipulation attacks that led to a total loss of 25 million dollars, PARTITIONGPT effectively prevents eight cases, highlighting its potential for broad applicability and the necessity for secure program partitioning during smart contract development to diminish manipulation vulnerabilities.
- 2 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (4) , 1549-1567
Towards Secure Program Partitioning for Smart Contracts With LLM’s In-Context Learning
Smart contracts are highly susceptible to manipulation attacks due to the leakage of sensitive information. Addressing manipulation vulnerabilities is particularly challenging because they stem from inherent data confidentiality issues rather than straightforward implementation bugs. To tackle this by preventing sensitive information leakage, we present PARTITIONGPT, the first LLM-driven approach that combines static analysis with the in-context learning capabilities of large language models (LLMs) to partition smart contracts into critical (privileged) and normal codebases, guided by a few annotated sensitive data variables. We evaluated PARTITIONGPT on 18 annotated smart contracts containing 99 sensitive functions. The results demonstrate that PARTITIONGPT successfully generates compilable, and verified partitions, achieving a precision of 80% while reducing more than 26% code compared to functionlevel partitioning approach. Furthermore, we evaluated PARTITIONGPT on nine real-world manipulation attacks that led to a total loss of 25 million dollars, PARTITIONGPT effectively prevents eight cases, highlighting its potential for broad applicability and the necessity for secure program partitioning during smart contract development to diminish manipulation vulnerabilities.
- Open Access
- 20 January 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (2) , 1-32
OptRCA: A More Efficient and Accurate Approach for Automated Root Cause Analysis and Explanation
With the development of automated software testing technology, software developers can get a large number of crash test cases in a short period of time. However, analyzing these crash test cases and finding their root cause is a time-consuming and labor-intensive task. Techniques based on reverse execution and backward taint analysis are proposed to locate the root cause, but can’t provide context information or explanation of the underlying fault. To address these two limitations, researchers have proposed an automated root cause analysis technique called AURORA. Although this technique provides powerful root cause analysis capabilities, it also have two obvious shortcomings. First, the results of root cause analysis are not accurate enough. Second, the efficiency of root cause analysis is not high enough. In order to improve these two shortcomings, we propose OptRCA, a more efficient and accurate approach for root cause analysis and explanation. Like AURORA’s fuzzing strategy, OptRCA is also designed based on AFL’s crash mode. The difference between them is mainly reflected in three points. First of all, the goal pursued by OptRCA is different from that of normal fuzzing technology. OptRCA pursues maximum correlation to ensure that as many crash test cases as possible are related to the same root cause. This test case with maximum correlation can greatly improve the accuracy of root cause analysis. Second, OptRCA proposed a more efficient non-crash test case retention strategy, which we named “Hill-Climbing Retention.” Using the hill-climbing retention method, OptRCA can obtain sufficient root cause information while retaining only a few non-crash test cases. Since the number of test cases is greatly reduced, the efficiency of OptRCA’s subsequent root cause analysis process is also greatly improved. In addition, OptRCA also optimizes the analysis formula to obtain more accurate analysis results. In the evaluation experimental results, OptRCA is significantly better than AURORA in terms of accuracy and efficiency. Quantitative analysis shows that OptRCA is 65% more accurate and 61% more efficient than AURORA.
- 20 January 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (2) , 1-32
Open AccessOptRCA: A More Efficient and Accurate Approach for Automated Root Cause Analysis and Explanation
With the development of automated software testing technology, software developers can get a large number of crash test cases in a short period of time. However, analyzing these crash test cases and finding their root cause is a time-consuming and labor-intensive task. Techniques based on reverse execution and backward taint analysis are proposed to locate the root cause, but can’t provide context information or explanation of the underlying fault. To address these two limitations, researchers have proposed an automated root cause analysis technique called AURORA. Although this technique provides powerful root cause analysis capabilities, it also have two obvious shortcomings. First, the results of root cause analysis are not accurate enough. Second, the efficiency of root cause analysis is not high enough. In order to improve these two shortcomings, we propose OptRCA, a more efficient and accurate approach for root cause analysis and explanation. Like AURORA’s fuzzing strategy, OptRCA is also designed based on AFL’s crash mode. The difference between them is mainly reflected in three points. First of all, the goal pursued by OptRCA is different from that of normal fuzzing technology. OptRCA pursues maximum correlation to ensure that as many crash test cases as possible are related to the same root cause. This test case with maximum correlation can greatly improve the accuracy of root cause analysis. Second, OptRCA proposed a more efficient non-crash test case retention strategy, which we named “Hill-Climbing Retention.” Using the hill-climbing retention method, OptRCA can obtain sufficient root cause information while retaining only a few non-crash test cases. Since the number of test cases is greatly reduced, the efficiency of OptRCA’s subsequent root cause analysis process is also greatly improved. In addition, OptRCA also optimizes the analysis formula to obtain more accurate analysis results. In the evaluation experimental results, OptRCA is significantly better than AURORA in terms of accuracy and efficiency. Quantitative analysis shows that OptRCA is 65% more accurate and 61% more efficient than AURORA.
- 30 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (3) , 770-785
Evaluating Large Language Models for Line-Level Vulnerability Localization
Recently, Automated Vulnerability Localization (AVL) has attracted growing attention, aiming to facilitate diagnosis by pinpointing the specific lines of code responsible for vulnerabilities. Large Language Models (LLMs) have shown potential in various domains, yet their effectiveness in line-level vulnerability localization remains underexplored. In this work, we present the first comprehensive empirical evaluation of LLMs for AVL. Our study examines 19 leading LLMs suitable for code analysis, including ChatGPT and multiple open-source models, spanning encoder-only, encoder-decoder, and decoder-only architectures, with model sizes from 60M to 70B parameters. We evaluate three paradigms—few-shot prompting, discriminative fine-tuning, and generative fine-tuning—with and without Low-Rank Adaptation (LoRA), on both a BigVul-derived dataset for C/C++ and a smart contract vulnerability dataset. Our results show that discriminative fine-tuning achieves substantial performance gains over existing learning-based AVL methods when sufficient training data is available. In low-data settings, prompting advanced LLMs such as ChatGPT proves more effective. We also identify challenges related to input length and unidirectional context during fine-tuning, and propose two remedial strategies: a sliding window approach and right-forward embedding, both of which yield significant improvements. Moreover, we provide the first assessment of LLM generalizability in AVL, showing that certain models can transfer effectively across Common Weakness Enumerations (CWEs) and projects. However, performance degrades notably for newly discovered vulnerabilities containing unfamiliar lexical or structural patterns, underscoring the need for continual adaptation. These findings offer practical guidance for deploying LLM-based AVL systems in realistic software security workflows.
- 30 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52 (3) , 770-785
Evaluating Large Language Models for Line-Level Vulnerability Localization
Recently, Automated Vulnerability Localization (AVL) has attracted growing attention, aiming to facilitate diagnosis by pinpointing the specific lines of code responsible for vulnerabilities. Large Language Models (LLMs) have shown potential in various domains, yet their effectiveness in line-level vulnerability localization remains underexplored. In this work, we present the first comprehensive empirical evaluation of LLMs for AVL. Our study examines 19 leading LLMs suitable for code analysis, including ChatGPT and multiple open-source models, spanning encoder-only, encoder-decoder, and decoder-only architectures, with model sizes from 60M to 70B parameters. We evaluate three paradigms—few-shot prompting, discriminative fine-tuning, and generative fine-tuning—with and without Low-Rank Adaptation (LoRA), on both a BigVul-derived dataset for C/C++ and a smart contract vulnerability dataset. Our results show that discriminative fine-tuning achieves substantial performance gains over existing learning-based AVL methods when sufficient training data is available. In low-data settings, prompting advanced LLMs such as ChatGPT proves more effective. We also identify challenges related to input length and unidirectional context during fine-tuning, and propose two remedial strategies: a sliding window approach and right-forward embedding, both of which yield significant improvements. Moreover, we provide the first assessment of LLM generalizability in AVL, showing that certain models can transfer effectively across Common Weakness Enumerations (CWEs) and projects. However, performance degrades notably for newly discovered vulnerabilities containing unfamiliar lexical or structural patterns, underscoring the need for continual adaptation. These findings offer practical guidance for deploying LLM-based AVL systems in realistic software security workflows.
- 17 October 2025
- Published by Springer Nature
- p. 489-494
An Analytical Perspective on Software Engineering for Large Language Models
- 17 October 2025
- Published by Springer Nature
- p. 489-494
An Analytical Perspective on Software Engineering for Large Language Models
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (11) , 3088-3102
An Empirical Study of Exploring the Capabilities of Large Language Models in Code Learning
Since the advent of ChatGPT, large language models (LLMs) have attracted widespread attention from academia and industry. They have also brought significant changes to software engineering. However, until now, there has been a lack of comprehensive studies comparing LLMs with previous smaller code pre-trained models. To address this gap, we conduct a study in this paper to illustrate the performance of LLMs in different software engineering tasks. Specifically, we select three open-source large language models, CodeGen, LLaMA, and StarCoder, for the research targets, and our study is conducted from four aspects, including code syntax understanding, code semantic reasoning, encoding representation quality, and adaptation performance for different software engineering tasks to compare LLMs with previous code pre-trained models. Four aspects build on each other, forming important components of AI for Software Engineering. We conclude that: (1) Compared with previous smaller pre-trained models like CodeBERT, LLMs exhibit distinct trends in how they learn code syntax or semantics as the number of layers increases. Additionally, mastering code semantics proves to be more challenging, with semantic information usually learned in the final layers; (2) Causal decoder architecture with left-to-right attention masking does not perform well in zero-shot tasks; (3) For classification tasks, the mean vector representation generated by LLMs over a sequence tends to outperform the last token representation in the sequence; (4) Incorporating parameter-efficient fine-tuning techniques into LLMs for downstream tasks can help LLMs achieve better performance than previous code pre-trained models on code generation tasks but may not be optimal in some code understanding tasks; (5) LoRA emerges as a more effective PEFT technique for LLMs in downstream code-related tasks. We hope these findings will better guide future researchers in designing more powerful code models.
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (11) , 3088-3102
An Empirical Study of Exploring the Capabilities of Large Language Models in Code Learning
Since the advent of ChatGPT, large language models (LLMs) have attracted widespread attention from academia and industry. They have also brought significant changes to software engineering. However, until now, there has been a lack of comprehensive studies comparing LLMs with previous smaller code pre-trained models. To address this gap, we conduct a study in this paper to illustrate the performance of LLMs in different software engineering tasks. Specifically, we select three open-source large language models, CodeGen, LLaMA, and StarCoder, for the research targets, and our study is conducted from four aspects, including code syntax understanding, code semantic reasoning, encoding representation quality, and adaptation performance for different software engineering tasks to compare LLMs with previous code pre-trained models. Four aspects build on each other, forming important components of AI for Software Engineering. We conclude that: (1) Compared with previous smaller pre-trained models like CodeBERT, LLMs exhibit distinct trends in how they learn code syntax or semantics as the number of layers increases. Additionally, mastering code semantics proves to be more challenging, with semantic information usually learned in the final layers; (2) Causal decoder architecture with left-to-right attention masking does not perform well in zero-shot tasks; (3) For classification tasks, the mean vector representation generated by LLMs over a sequence tends to outperform the last token representation in the sequence; (4) Incorporating parameter-efficient fine-tuning techniques into LLMs for downstream tasks can help LLMs achieve better performance than previous code pre-trained models on code generation tasks but may not be optimal in some code understanding tasks; (5) LoRA emerges as a more effective PEFT technique for LLMs in downstream code-related tasks. We hope these findings will better guide future researchers in designing more powerful code models.