What You See is What You Get: Attention-Based Self-Guided Automatic Unit Test Generation
Authors:
- 26 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1039-1051
Abstract
Software defects heavily affect software's function-alities and may cause huge losses. Recently, many AI-based approaches have been proposed to detect defects, which can be divided into two categories: software defect prediction and automatic unit test generation. While these approaches have made great progress in software defect detection, they still have several limitations in practical application, including the low confidence of prediction models and the inefficiency of unit testing models. To address these limitations, we propose a WYSIWYG (i.e., What You See Is What You Get) approach: Attention-based Self-guided Automatic Unit Test GenERation (AUGER), which contains two stages: defect detection and error triggering. In the former stage, AUGER first detects the proneness of defects. Then, in the latter stage, it guides to generate unit tests for triggering such an error with the help of critical information obtained by the former stage. To evaluate the effectiveness of AUGER, we conduct a large-scale experiment by comparing with the state-of-the-art (SOTA) approaches on the widely used datasets (i.e., Bears, Bugs.jar, and Defects4J). AUGER makes great improvements by 4.7% to 35.3% and 17.7% to 40.4% in terms of Fl-score and Precision in defect detection, and can trigger 23 to 84 more errors than SOTAs in unit test generation. Besides, we also conduct a further study to verify the generalization in practical usage by collecting a new dataset from real-world projects.Keywords
Funding Information
- National Natural Science Foundation of China (62202419)
- Fundamental Research Funds for the Central Universities (226-2022-00064)
- Ningbo Natural Science Foundation (2022JI84)
This publication has 56 references indexed in Scilit:
- A3Test: Assertion-Augmented Automated Test case generationInformation and Software Technology, 2024
- PyExplainer: Explaining the Predictions of Just-In-Time Defect Models2021
- An Industrial Evaluation of Unit Test Generation: Finding Real Faults in a Financial Application2017
- Analysing the fitness landscape of search-based software testing problemsAutomated Software Engineering, 2016
- Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges (T)2015
- Studying high impact fix-inducing changesEmpirical Software Engineering, 2015
- Why don't software developers use static analysis tools to find bugs?2013
- Predicting Faults from Cached History29th International Conference on Software Engineering (ICSE'07), 2007
- Use of relative code churn measures to predict system defect density2005
- Predicting fault incidence using software change historyIEEE Transactions on Software Engineering, 2000