Combining Language and App Ui Analysis for the Automated Assessment of Bug Reproduction Steps
Authors:
- 27 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-12
Abstract
Bug reports are essential for developers to confirm software problems, investigate their causes, and validate fixes. Unfortunately, reports often miss important information or are written unclearly, which can cause delays, increased issue resolution effort, or even the inability to solve issues. One of the most common components of reports that are problematic is the steps to reproduce the bug(s) (S2Rs), which are essential to replicate the described program failures and reason about fixes. Given the proclivity for deficiencies in reported S2Rs, prior work has proposed techniques that assist reporters in writing or assessing the quality of S2Rs. However, automated understanding of S2Rs is challenging, and requires linking nuanced natural language phrases with specific, semantically related program information. Prior techniques often struggle to form such language $\leftrightarrow$ program connections - due to issues in language variability and limitations of information gleaned from program analyses. To more effectively tackle the problem of S2R quality annotation, we propose a new technique called AstroBR, which leverages the language understanding capabilities of LLMs to identify and extract the S2Rs from bug reports and map them to GUI interactions in a program state model derived via dynamic analysis. We compared AstroBR to a related state-of-the-art approach and we found that AstroBR annotates S2Rs 25.2% better (in terms of F1 score) than the baseline. Additionally, AstroBR suggests more accurate missing S2Rs than the baseline (by 71.4 % in terms of F1 score).
Keywords
All Related Versions
Funding Information
- U.S. NSF (CCF-1955853,CCF-2343057,CCF-2441355)
This publication has 51 references indexed in Scilit:
- Burt: A Chatbot for Interactive Bug Reporting2023
- V2S: A Tool for Translating Video Recordings of Mobile App Usages into Replayable Scenarios2021
- It Takes Two to Tango: Combining Visual and Textual Information for Detecting Duplicate Video-Based Bug Reports2021
- Not all bugs are the same: Understanding, characterizing, and classifying bug typesJournal of Systems and Software, 2019
- ReCDroid: Automatically Reproducing Android Application Crashes from Bug Reports2019
- Using bug descriptions to reformulate queries during text-retrieval-based bug localizationEmpirical Software Engineering, 2019
- CrashScope: A Practical Tool for Automated Testing of Android Applications2017
- On the Vocabulary Agreement in Software Issue Descriptions2016
- Automated prediction of bug report priority using multi-factor analysisEmpirical Software Engineering, 2014
- Characterizing and predicting which bugs get reopened2012