Human-Machine Co-Boosted Bug Report Identification with Mutualistic Neural Active Learning
Open Access
- 27 April 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Abstract
Bug reports, encompassing a wide range of bug types, are crucial for maintaining software quality. However, the increasing complexity and volume of bug reports pose a significant challenge in sole manual identification and assignment to the appropriate teams for resolution, as dealing with all the reports is time-consuming and resource-intensive. In this paper, we introduce a cross-project framework, dubbed Mutualistic Neural Active Learning ( MNAL ), designed for automated and more effective identification of bug reports from GitHub repositories boosted by human-machine collaboration. MNAL utilizes a neural language model that learns and generalizes reports across different projects, coupled with active learning to form neural active learning. A distinctive feature of MNAL is the purposely crafted mutualistic relation between the machine learners (neural language model) and human labelers (developers) when enriching the knowledge learned. That is, the most informative human-labeled reports and their corresponding pseudo-labeled ones are used to update the model while those reports that need to be labeled by developers are more readable and identifiable, thereby enhancing the human-machine teaming therein. We evaluate MNAL using a dataset of \(1,275,881\) reports from over \(127,000\) software projects against the state-of-the-art approaches, baselines, and different variants. The results indicate that, remarkably, MNAL achieves up to 95.8% and 196.0% effort reduction in terms of readability and identifiability during human labeling, respectively, while resulting in a better performance (e.g., F1-score) in bug report identification. Additionally, our MNAL is model-agnostic since it is capable of improving the model performance with various underlying neural language models. To further verify the efficacy of our approach, we conducted a qualitative case study involving 10 human participants, who rate MNAL as being more effective while saving more time and monetary resources. The dataset and code are made publicly available at https://github.com/ideas-labo/MNAL .
Keywords
All Related Versions
This publication has 87 references indexed in Scilit:
- Knowledge-Driven Active Learning2023
- Automatic Classification of Bug Reports Based on Multiple Text Information and Reports’ Intention2022
- Prediction of bug‐fixing time based on distinguishable sequences fusion in open source softwareJournal of Software: Evolution and Process, 2022
- Towards the identification of bug entities and relations in bug reportsAutomated Software Engineering, 2022
- A method of non-bug report identification from bug report repositoryArtificial Life and Robotics, 2021
- On the classification of bug reports to improve bug localizationSoft Computing, 2021
- BugTracking: A Tool to Assist in the Identification of Bug Reports2016
- Combining text mining and data mining for bug report classificationJournal of Software: Evolution and Process, 2016
- A practical guide to controlled experiments of software engineering tools with human participantsEmpirical Software Engineering, 2013
- A survey on instance selection for active learningKnowledge and Information Systems, 2012