An Guo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
An Guo has a total of 74 co-authors. Here are the top co-authors by number of shared publications:
- Chunrong Fang
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
12 shared publications - Zhenyu Chen
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
12 shared publications - Haoxiang Tian
Nanyang Technological University, College of Computing and Data Science, Singapore, Singapore
7 shared publications - Xinyu Gao
Nanjing University, Nanjing, China
7 shared publications - Weisong Sun
Nanyang technological university, Singapore, Singapore
6 shared publications - Enyi Tang
Nanjing University, State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
4 shared publications
Latest Publications
- Open Access
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3101-3113
TrackTest: Trajectory Generation and Prioritization for Testing Multiple Object Tracking System
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3101-3113
Open AccessTrackTest: Trajectory Generation and Prioritization for Testing Multiple Object Tracking System
- Open Access
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3  (ISSTA) , 1997-2020
StateTree: A Tree-Based Modeling Approach for Fault Detection in Recurrent Neural Networks
Recurrent Neural Networks (RNNs) have become a core component of modern intelligent software due to their strong ability to model temporal dependencies. As RNNs are increasingly deployed in safety-critical domains, ensuring their reliability is crucial. However, most existing testing techniques are designed for feedforward networks and struggle with RNNs. The stateful nature, recurrent feedback, and long-term dependencies of RNNs make it difficult for existing testing methods to capture decision logic and temporal behaviors, which in turn makes detecting fault-inducing behaviors that emerge through temporal decision evolution challenging. To address these challenges, we propose StateTree, a tree-based abstract modeling approach for systematic testing of RNN-based systems. StateTree constructs an Abstract State Tree (AST) that captures major RNN decision behaviors, where each root-to-leaf path represents an abstract decision for intuitive interpretation and structured exploration. Using the AST, StateTree guides the testing process toward both major and previously unseen paths to reveal erroneous behaviors. Experiments show that StateTree accurately abstracts RNN decisions, detects hundreds of faults, and retraining with its identified test cases improves robustness beyond existing RNN coverage methods, demonstrating its effectiveness in both fault detection and model performance enhancement.
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3  (ISSTA) , 1997-2020
Open AccessStateTree: A Tree-Based Modeling Approach for Fault Detection in Recurrent Neural Networks
Recurrent Neural Networks (RNNs) have become a core component of modern intelligent software due to their strong ability to model temporal dependencies. As RNNs are increasingly deployed in safety-critical domains, ensuring their reliability is crucial. However, most existing testing techniques are designed for feedforward networks and struggle with RNNs. The stateful nature, recurrent feedback, and long-term dependencies of RNNs make it difficult for existing testing methods to capture decision logic and temporal behaviors, which in turn makes detecting fault-inducing behaviors that emerge through temporal decision evolution challenging. To address these challenges, we propose StateTree, a tree-based abstract modeling approach for systematic testing of RNN-based systems. StateTree constructs an Abstract State Tree (AST) that captures major RNN decision behaviors, where each root-to-leaf path represents an abstract decision for intuitive interpretation and structured exploration. Using the AST, StateTree guides the testing process toward both major and previously unseen paths to reveal erroneous behaviors. Experiments show that StateTree accurately abstracts RNN decisions, detects hundreds of faults, and retraining with its identified test cases improves robustness beyond existing RNN coverage methods, demonstrating its effectiveness in both fault detection and model performance enhancement.
- Open Access
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3  (FSE) , 2720-2743
OdoTest: An Automated Testing Approach for Odometry Systems
Reliable pose estimation is critical for intelligent systems, including autonomous vehicles, unmanned aerial vehicles, and virtual reality applications. While visual-inertial odometry (VIO) has made significant advancements in estimating pose, its performance can still be affected by sensor noise, environmental variations, and calibration errors. To evaluate the performance of VIO, existing testing methods rely on real-world datasets or manually degraded data, where sensor measurements or images are artificially modified according to predefined, fixed perturbation patterns. These approaches are costly, require time-consuming annotation, and rely on ad hoc modifications that limit scenario diversity and hinder the exploration of challenging cases. In this paper, we design and implement OdoTest, the automated testing framework for VIO systems. OdoTest equips sensor-specific transformation operators, including IMU perturbations, camera degradations, and sensor calibration errors, that systematically generate various and realistic test scenarios. Moreover, OdoTest adopts an odometry fitness-guided testing strategy to prioritize scenario generation, improving testing efficiency. By leveraging odometry-specific metamorphic relations (MRs), OdoTest can automatically detect odometry errors without requiring manual ground-truth labels for test scenarios. We evaluate OdoTest on twelve state-of-the-art VIO systems to answer three main research questions: 1) the effectiveness of transformation operators in revealing odometry errors; 2) the ability of OdoTest to generate error-triggering scenarios; and 3) whether retraining on these scenarios can improve odometry performance. Results show that OdoTest’s transformation operators effectively expose odometry errors, while its fitness-guided scenario generation efficiently creates challenging test cases. Moreover, retraining with these scenarios significantly enhances both estimation accuracy and system robustness, which demonstrates the usefulness of OdoTest.
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3  (FSE) , 2720-2743
Open AccessOdoTest: An Automated Testing Approach for Odometry Systems
Reliable pose estimation is critical for intelligent systems, including autonomous vehicles, unmanned aerial vehicles, and virtual reality applications. While visual-inertial odometry (VIO) has made significant advancements in estimating pose, its performance can still be affected by sensor noise, environmental variations, and calibration errors. To evaluate the performance of VIO, existing testing methods rely on real-world datasets or manually degraded data, where sensor measurements or images are artificially modified according to predefined, fixed perturbation patterns. These approaches are costly, require time-consuming annotation, and rely on ad hoc modifications that limit scenario diversity and hinder the exploration of challenging cases. In this paper, we design and implement OdoTest, the automated testing framework for VIO systems. OdoTest equips sensor-specific transformation operators, including IMU perturbations, camera degradations, and sensor calibration errors, that systematically generate various and realistic test scenarios. Moreover, OdoTest adopts an odometry fitness-guided testing strategy to prioritize scenario generation, improving testing efficiency. By leveraging odometry-specific metamorphic relations (MRs), OdoTest can automatically detect odometry errors without requiring manual ground-truth labels for test scenarios. We evaluate OdoTest on twelve state-of-the-art VIO systems to answer three main research questions: 1) the effectiveness of transformation operators in revealing odometry errors; 2) the ability of OdoTest to generate error-triggering scenarios; and 3) whether retraining on these scenarios can improve odometry performance. Results show that OdoTest’s transformation operators effectively expose odometry errors, while its fitness-guided scenario generation efficiently creates challenging test cases. Moreover, retraining with these scenarios significantly enhances both estimation accuracy and system robustness, which demonstrates the usefulness of OdoTest.
- Open Access
- 19 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Understanding LLM-Centric Challenges for Deep Learning Frameworks: An Empirical Analysis
Large language models (LLMs) have achieved remarkable success across diverse applications, but their development and deployment depend heavily on deep learning (DL) frameworks. The massive scale, long execution cycles, and complex workflows of LLMs place stringent demands on framework usability, functionality, and reliability. When these demands are not well supported, developers may suffer from reduced efficiency, severe failures, and substantial resource waste. However, a fundamental question remains underexplored: what challenges do DL frameworks face in supporting LLMs? To answer this question, we analyze issue reports from three major DL frameworks (i.e., MindSpore, PyTorch, and TensorFlow) and eight associated LLM toolkits such as DeepSpeed. Based on a manual review of these reports, we construct a taxonomy that captures LLM-centric framework bugs, user requirements, and user questions. We then refine and enrich this taxonomy through interviews with 11 LLM users and eight DL framework developers. Using the taxonomy and interview findings, our study further reveals key technical challenges and mismatches between LLM user needs and developer priorities. Overall, our contributions are threefold: (1) we develop a comprehensive taxonomy comprising five question themes (11 sub-themes), five requirement themes (17 sub-themes), and ten bug themes (47 sub-themes); (2) we assess the importance and priority of different categories in the taxonomy based on insights from LLM users and framework developers; and (3) we identify five key findings across the LLM development lifecycle and propose five actionable recommendations to improve the quality of DL frameworks in supporting LLMs. Our results highlight critical limitations in current DL frameworks and offer concrete guidance for advancing their support for future LLM development, deployment, and applications.
- 19 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessUnderstanding LLM-Centric Challenges for Deep Learning Frameworks: An Empirical Analysis
Large language models (LLMs) have achieved remarkable success across diverse applications, but their development and deployment depend heavily on deep learning (DL) frameworks. The massive scale, long execution cycles, and complex workflows of LLMs place stringent demands on framework usability, functionality, and reliability. When these demands are not well supported, developers may suffer from reduced efficiency, severe failures, and substantial resource waste. However, a fundamental question remains underexplored: what challenges do DL frameworks face in supporting LLMs? To answer this question, we analyze issue reports from three major DL frameworks (i.e., MindSpore, PyTorch, and TensorFlow) and eight associated LLM toolkits such as DeepSpeed. Based on a manual review of these reports, we construct a taxonomy that captures LLM-centric framework bugs, user requirements, and user questions. We then refine and enrich this taxonomy through interviews with 11 LLM users and eight DL framework developers. Using the taxonomy and interview findings, our study further reveals key technical challenges and mismatches between LLM user needs and developer priorities. Overall, our contributions are threefold: (1) we develop a comprehensive taxonomy comprising five question themes (11 sub-themes), five requirement themes (17 sub-themes), and ten bug themes (47 sub-themes); (2) we assess the importance and priority of different categories in the taxonomy based on insights from LLM users and framework developers; and (3) we identify five key findings across the LLM development lifecycle and propose five actionable recommendations to improve the quality of DL frameworks in supporting LLMs. Our results highlight critical limitations in current DL frameworks and offer concrete guidance for advancing their support for future LLM development, deployment, and applications.
- Open Access
- 6 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Abstract Syntax Tree for Programming Language Understanding and Representation
Programming language understanding and representation (a.k.a code representation learning) has always been a hot and challenging task in the field of software engineering. It aims to apply deep learning techniques to produce numerical representations of the source code features while preserving its semantics. These representations can facilitate subsequent code-related tasks, e.g., code summarization. The abstract syntax tree (AST), a fundamental code feature, illustrates the syntactic information of the source code and has been widely used in code representation learning. It is commonly acknowledged that AST-based code representation is critical to solving code-related tasks. However, there is still a lack of systematic and quantitative evaluation of how well AST-based code representation facilitates subsequent code-related tasks. Additionally, learning an AST-based code representation is an extremely complex endeavor involving three intertwining stages, including AST parsing, AST preprocessing, and AST encoding. The solutions available in each stage are diverse. There is currently a lack of guidance on selecting solutions at each stage to get the most out of AST. In this paper, we first conduct comprehensive experiments to reveal the impact of the choice of AST parsing/preprocessing/encoding methods on AST-based code representation across three popular types of code-related tasks, including code clone detection, code search, and code summarization. The experiments involve four AST parsing methods, six AST preprocessing methods, and four AST encoding methods, all of which are widely utilized in existing AST-based code representation research. The experimental results showcase that the impact of different methods at different stages varies for different code-related tasks. Based on these, we further explore the practical influence of the AST-based code representation in facilitating follow-up code-related tasks. To do so, we compare the performance of models trained with code token sequence (Token for short) based code representation and AST-based code representation on the three code-related tasks. Surprisingly, the overall quantitative statistical results demonstrate that models trained with Token-based code representation consistently perform better across all three tasks compared to models trained with AST-based code representation. Our further quantitative analysis reveals that models trained with AST-based code representation outperform models trained with Token-based code representation in certain subsets of samples across all three tasks. For instance, in the code summarization task, such samples constitute as much as 39% of the total, while in the code search task, they account for 28%. As ASTs are now being used in practice under various contexts (a.k.a., code-related tasks), the results in this paper call for more research on context-specific AST-based code representation learning in the future. Our study provides future researchers with detailed guidance on how to select solutions at each stage to fully exploit AST.
- 6 June 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessAbstract Syntax Tree for Programming Language Understanding and Representation
Programming language understanding and representation (a.k.a code representation learning) has always been a hot and challenging task in the field of software engineering. It aims to apply deep learning techniques to produce numerical representations of the source code features while preserving its semantics. These representations can facilitate subsequent code-related tasks, e.g., code summarization. The abstract syntax tree (AST), a fundamental code feature, illustrates the syntactic information of the source code and has been widely used in code representation learning. It is commonly acknowledged that AST-based code representation is critical to solving code-related tasks. However, there is still a lack of systematic and quantitative evaluation of how well AST-based code representation facilitates subsequent code-related tasks. Additionally, learning an AST-based code representation is an extremely complex endeavor involving three intertwining stages, including AST parsing, AST preprocessing, and AST encoding. The solutions available in each stage are diverse. There is currently a lack of guidance on selecting solutions at each stage to get the most out of AST. In this paper, we first conduct comprehensive experiments to reveal the impact of the choice of AST parsing/preprocessing/encoding methods on AST-based code representation across three popular types of code-related tasks, including code clone detection, code search, and code summarization. The experiments involve four AST parsing methods, six AST preprocessing methods, and four AST encoding methods, all of which are widely utilized in existing AST-based code representation research. The experimental results showcase that the impact of different methods at different stages varies for different code-related tasks. Based on these, we further explore the practical influence of the AST-based code representation in facilitating follow-up code-related tasks. To do so, we compare the performance of models trained with code token sequence (Token for short) based code representation and AST-based code representation on the three code-related tasks. Surprisingly, the overall quantitative statistical results demonstrate that models trained with Token-based code representation consistently perform better across all three tasks compared to models trained with AST-based code representation. Our further quantitative analysis reveals that models trained with AST-based code representation outperform models trained with Token-based code representation in certain subsets of samples across all three tasks. For instance, in the code summarization task, such samples constitute as much as 39% of the total, while in the code search task, they account for 28%. As ASTs are now being used in practice under various contexts (a.k.a., code-related tasks), the results in this paper call for more research on context-specific AST-based code representation learning in the future. Our study provides future researchers with detailed guidance on how to select solutions at each stage to fully exploit AST.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 3298-3310
Scalpel: Automotive Deep Learning Framework Testing via Assembling Model Components
Deep learning (DL) plays a key role in autonomous driving systems. DL models support perception modules, equipped with tasks such as object detection and sensor fusion. These DL models enable vehicles to process multi-sensor inputs to understand complex surroundings. Deploying DL models in autonomous driving systems faces stringent challenges, including real-time processing, limited computational resources, and strict power constraints. To address these challenges, automotive DL frameworks (e.g., PaddleInference) have emerged to optimize inference efficiency. However, these frameworks encounter unique quality issues due to their more complex deployment environments, such as crashes stemming from limited scheduled memory and incorrect memory allocation. Unfortunately, existing DL framework testing methods fail to detect these quality issues due to the failure in deploying generated test input models, as these models lack three essential capabilities: (1) multi-input/output tensor processing, (2) multi-modal data processing, and (3) multi-level data feature extraction. These capabilities necessitate specialized model components, which existing testing methods neglect during model generation. To bridge this gap, we propose Scalpel, an automotive DL frameworks testing method that generates test input models at the model component level. Scalpel generates models by assembling model components (heads, necks, backbones) to support capabilities required by autonomous driving systems. Specifically, Scalpel maintains and updates a repository of model components, generating test inputs by selecting, mutating, and assembling them. Successfully generated models are added back to enrich the repository. Newly generated models are then deployed within the autonomous driving system to test automotive DL frameworks via differential testing. The experimental results demonstrate that Scalpel outperforms existing methods in both effectiveness and efficiency. In Apollo, Scalpel detects 16 crashes and 21 NaN & inconsistency bugs. All detected bugs have been reported to open-source communities, with 10 crashes confirmed. Scalpel achieves 27.44 × and 8.5 × improvements in model generation efficiency and bug detection efficiency. Additionally, Scalpel detects nine crashes and 16 NaN & inconsistency bugs in Autoware, which shows its excellent generalization.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 3298-3310
Open AccessScalpel: Automotive Deep Learning Framework Testing via Assembling Model Components
Deep learning (DL) plays a key role in autonomous driving systems. DL models support perception modules, equipped with tasks such as object detection and sensor fusion. These DL models enable vehicles to process multi-sensor inputs to understand complex surroundings. Deploying DL models in autonomous driving systems faces stringent challenges, including real-time processing, limited computational resources, and strict power constraints. To address these challenges, automotive DL frameworks (e.g., PaddleInference) have emerged to optimize inference efficiency. However, these frameworks encounter unique quality issues due to their more complex deployment environments, such as crashes stemming from limited scheduled memory and incorrect memory allocation. Unfortunately, existing DL framework testing methods fail to detect these quality issues due to the failure in deploying generated test input models, as these models lack three essential capabilities: (1) multi-input/output tensor processing, (2) multi-modal data processing, and (3) multi-level data feature extraction. These capabilities necessitate specialized model components, which existing testing methods neglect during model generation. To bridge this gap, we propose Scalpel, an automotive DL frameworks testing method that generates test input models at the model component level. Scalpel generates models by assembling model components (heads, necks, backbones) to support capabilities required by autonomous driving systems. Specifically, Scalpel maintains and updates a repository of model components, generating test inputs by selecting, mutating, and assembling them. Successfully generated models are added back to enrich the repository. Newly generated models are then deployed within the autonomous driving system to test automotive DL frameworks via differential testing. The experimental results demonstrate that Scalpel outperforms existing methods in both effectiveness and efficiency. In Apollo, Scalpel detects 16 crashes and 21 NaN & inconsistency bugs. All detected bugs have been reported to open-source communities, with 10 crashes confirmed. Scalpel achieves 27.44 × and 8.5 × improvements in model generation efficiency and bug detection efficiency. Additionally, Scalpel detects nine crashes and 16 NaN & inconsistency bugs in Autoware, which shows its excellent generalization.
- Open Access
- 19 January 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35  (9) , 1-44
Discovering Safety Violations of Decision-Making in Autonomous Driving Systems from Accident-Free Traffic Scenarios
Safety testing serves as the fundamental pillar for the development of autonomous driving systems (ADSs), and decision-making plays a key role in ADSs. To ensure the safety of ADSs, it is paramount to generate a range of critical test scenarios to test the safety of decision-making in ADSs. While existing researches primarily focus on reproducing real-world traffic accidents in simulation environments to create test scenarios, it is essential to highlight that many of these accidents do not result in safety violations of decision-making in ADSs due to the differences between human driving and autonomous driving. More importantly, we observe that some accident-free real-world scenarios can lead to misbehaviors of ADSs. Therefore, orthogonally to existing work, it is equally important to discover safety violations of ADSs from routine traffic scenarios (i.e., accident-free scenarios) to ensure the safety of Autonomous Vehicles (AVs). We introduce CRISER , a novel methodology to achieve the above goal. It automatically generates abstract and concrete scenarios from real-traffic videos where human-driving worked safely. Based on them, CRISER discovers safety violations of the ADS's decision-making in semantic equivalent scenarios (i.e., the test scenarios with the same semantics as the original accident-free traffic videos). Specifically, CRISER enhances the ability of Large Multimodal Models (LMMs) to accurately extract scenario semantics from accident-free traffic videos and generate test scenarios by multi-modal few-shot Chain of Thought (CoT). Based on them, CRISER explores the behavior differences between the ego vehicle (i.e., the vehicle connected to the ADS under test) and human-driving in semantic equivalent scenarios. During the exploration search, CRISER keeps the semantic consistency of test scenarios with accident-free traffic videos, and explores the universality of discovered safety violations of the ADS. We implement and evaluate CRISER on the industrial-grade Level-4 ADS, Apollo. The experimental results demonstrate that CRISER can accurately extract scenario semantics and generate test scenarios from traffic videos, and effectively discover distinct types of safety violations of Apollo’s decision-making in accident-free traffic scenarios.
- 19 January 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35  (9) , 1-44
Open AccessDiscovering Safety Violations of Decision-Making in Autonomous Driving Systems from Accident-Free Traffic Scenarios
Safety testing serves as the fundamental pillar for the development of autonomous driving systems (ADSs), and decision-making plays a key role in ADSs. To ensure the safety of ADSs, it is paramount to generate a range of critical test scenarios to test the safety of decision-making in ADSs. While existing researches primarily focus on reproducing real-world traffic accidents in simulation environments to create test scenarios, it is essential to highlight that many of these accidents do not result in safety violations of decision-making in ADSs due to the differences between human driving and autonomous driving. More importantly, we observe that some accident-free real-world scenarios can lead to misbehaviors of ADSs. Therefore, orthogonally to existing work, it is equally important to discover safety violations of ADSs from routine traffic scenarios (i.e., accident-free scenarios) to ensure the safety of Autonomous Vehicles (AVs). We introduce CRISER , a novel methodology to achieve the above goal. It automatically generates abstract and concrete scenarios from real-traffic videos where human-driving worked safely. Based on them, CRISER discovers safety violations of the ADS's decision-making in semantic equivalent scenarios (i.e., the test scenarios with the same semantics as the original accident-free traffic videos). Specifically, CRISER enhances the ability of Large Multimodal Models (LMMs) to accurately extract scenario semantics from accident-free traffic videos and generate test scenarios by multi-modal few-shot Chain of Thought (CoT). Based on them, CRISER explores the behavior differences between the ego vehicle (i.e., the vehicle connected to the ADS under test) and human-driving in semantic equivalent scenarios. During the exploration search, CRISER keeps the semantic consistency of test scenarios with accident-free traffic videos, and explores the universality of discovered safety violations of the ADS. We implement and evaluate CRISER on the industrial-grade Level-4 ADS, Apollo. The experimental results demonstrate that CRISER can accurately extract scenario semantics and generate test scenarios from traffic videos, and effectively discover distinct types of safety violations of Apollo’s decision-making in accident-free traffic scenarios.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52  (10) , 1-18
Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52  (10) , 1-18
Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models
- Open Access
- 18 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35  (8) , 1-29
Pre-trained Model-based Actionable Warning Identification: A Feasibility Study
Actionable Warning Identification (AWI) plays a pivotal role in improving the usability of Static Code Analyzers (SCAs). Currently, Machine Learning (ML)-based AWI approaches, which mainly learn an AWI classifier from labeled warnings, are notably common. However, these approaches still face the problem of restricted performance due to the direct reliance on a limited number of labeled warnings to develop a classifier. Very recently, Pre-Trained Models (PTMs), which have been trained through billions of text/code tokens and have demonstrated substantial successful applications in various code-related tasks, could potentially address the above problem. Nevertheless, the performance of PTMs on AWI has not been systematically investigated, leaving a gap in understanding their pros and cons. In this paper, we are the first to explore the feasibility of applying various PTMs for AWI. By conducting an extensive evaluation on 12K+ warnings involving four commonly used SCAs (i.e., SpotBugs, Infer, CppCheck, and CSA) and three typical programming languages (i.e., Java, C, and C++), we (1) investigate the overall PTM-based AWI performance compared to the state-of-the-art ML-based AWI approach, (2) analyze the impact of three primary aspects (i.e., data preprocessing, model training, and model prediction) in the typical PTM-based AWI workflow, and (3) identify the reasons for the current underperformance of PTMs on AWI, thereby obtaining a series of findings. Based on the above findings, we further provide several potential directions to enhance PTM-based AWI.
- 18 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35  (8) , 1-29
Open AccessPre-trained Model-based Actionable Warning Identification: A Feasibility Study
Actionable Warning Identification (AWI) plays a pivotal role in improving the usability of Static Code Analyzers (SCAs). Currently, Machine Learning (ML)-based AWI approaches, which mainly learn an AWI classifier from labeled warnings, are notably common. However, these approaches still face the problem of restricted performance due to the direct reliance on a limited number of labeled warnings to develop a classifier. Very recently, Pre-Trained Models (PTMs), which have been trained through billions of text/code tokens and have demonstrated substantial successful applications in various code-related tasks, could potentially address the above problem. Nevertheless, the performance of PTMs on AWI has not been systematically investigated, leaving a gap in understanding their pros and cons. In this paper, we are the first to explore the feasibility of applying various PTMs for AWI. By conducting an extensive evaluation on 12K+ warnings involving four commonly used SCAs (i.e., SpotBugs, Infer, CppCheck, and CSA) and three typical programming languages (i.e., Java, C, and C++), we (1) investigate the overall PTM-based AWI performance compared to the state-of-the-art ML-based AWI approach, (2) analyze the impact of three primary aspects (i.e., data preprocessing, model training, and model prediction) in the typical PTM-based AWI workflow, and (3) identify the reasons for the current underperformance of PTMs on AWI, thereby obtaining a series of findings. Based on the above findings, we further provide several potential directions to enhance PTM-based AWI.
- 31 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52  (1) , 187-205
Spatial Semantic Fuzzing for LiDAR-Based Autonomous Driving Perception Systems
Autonomous driving systems (ADSs) have the potential to enhance safety through advanced perception and reaction capabilities, reduce emissions by alleviating congestion, and contribute to various improvements in quality of life. Despite significant advancements in ADSs, several real-world accidents resulting in fatalities have occurred due to failures in the autonomous driving perception modules. As a critical component of autonomous vehicles, LiDAR-based perception systems are marked by high complexity and low interpretability, necessitating the development of effective testing methods for these systems. Current testing methods largely depend on manual data collection and labeling, which restricts their ability to detect a diverse range of erroneous behaviors. This process is not only time-consuming and labor-intensive, but it may also result in the recurrent discovery of similar erroneous behaviors during testing, hindering a comprehensive assessment of the systems. In this paper, we propose and implement a fuzzing framework for LiDAR-based autonomous driving perception systems, named LDFuzz, grounded in metamorphic testing theory. This framework offers the first uniform solution for the automated generation of tests with oracle information. To enhance testing efficiency and increase the number of tests that identify erroneous behaviors, we incorporate spatial and semantic coverage based on the characteristics of point cloud data to guide the generation process. We evaluate the performance of LDFuzz through experiments conducted on four LiDAR-based autonomous driving perception systems designed for the 3D object detection task. The experimental results demonstrate that the tests produced by LDFuzz can effectively detect an average of 7.5% more erroneous behaviors within LiDAR-based perception systems than the optimal baseline. Furthermore, the findings indicate that LDFuzz significantly enhances the diversity of failed tests.
- 31 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 52  (1) , 187-205
Spatial Semantic Fuzzing for LiDAR-Based Autonomous Driving Perception Systems
Autonomous driving systems (ADSs) have the potential to enhance safety through advanced perception and reaction capabilities, reduce emissions by alleviating congestion, and contribute to various improvements in quality of life. Despite significant advancements in ADSs, several real-world accidents resulting in fatalities have occurred due to failures in the autonomous driving perception modules. As a critical component of autonomous vehicles, LiDAR-based perception systems are marked by high complexity and low interpretability, necessitating the development of effective testing methods for these systems. Current testing methods largely depend on manual data collection and labeling, which restricts their ability to detect a diverse range of erroneous behaviors. This process is not only time-consuming and labor-intensive, but it may also result in the recurrent discovery of similar erroneous behaviors during testing, hindering a comprehensive assessment of the systems. In this paper, we propose and implement a fuzzing framework for LiDAR-based autonomous driving perception systems, named LDFuzz, grounded in metamorphic testing theory. This framework offers the first uniform solution for the automated generation of tests with oracle information. To enhance testing efficiency and increase the number of tests that identify erroneous behaviors, we incorporate spatial and semantic coverage based on the characteristics of point cloud data to guide the generation process. We evaluate the performance of LDFuzz through experiments conducted on four LiDAR-based autonomous driving perception systems designed for the 3D object detection task. The experimental results demonstrate that the tests produced by LDFuzz can effectively detect an average of 7.5% more erroneous behaviors within LiDAR-based perception systems than the optimal baseline. Furthermore, the findings indicate that LDFuzz significantly enhances the diversity of failed tests.