Bug Classification in Quantum Software: A Rule-Based Framework and Its Evaluation
Preprint
Abstract
Accurate classification of software bugs is essential for improving software quality. This paper presents a rule-based automated framework for classifying issues in quantum software repositories by bug type, category, severity, and impacted quality attributes, with additional focus on quantum-specific bug types. The framework applies keyword and heuristic-based techniques tailored to quantum computing. To assess its reliability, we manually classified a stratified sample of 4,984 issues from a dataset of 12,910 issues across 36 Qiskit repositories. Automated classifications were compared with ground truth using accuracy, precision, recall, and F1-score. The framework achieved up to 85.21% accuracy, with F1-scores ranging from 0.7075 (severity) to 0.8393 (quality attribute). Statistical validation via paired t-tests and Cohen's Kappa showed substantial to almost perfect agreement for bug type (k = 0.696), category (k = 0.826), quality attribute (k = 0.818), and quantum-specific bug type (k = 0.712). Severity classification showed slight agreement (k = 0.162), suggesting room for improvement. Large-scale analysis revealed that classical bugs dominate (67.2%), with quantum-specific bugs at 27.3%. Frequent bug categories included compatibility, functional, and quantum-specific defects, while usability, maintainability, and interoperability were the most impacted quality attributes. Most issues (93.7%) were low severity; only 4.3% were critical. A detailed review of 1,550 quantum-specific bugs showed that over half involved quantum circuit-level problems, followed by gate errors and hardware-related issues.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Bugs4Q: A benchmark of existing bugs to enable controlled testing and debugging studies for quantum programsJournal of Systems and Software, 2023
- A comprehensive empirical study on bug characteristics of deep learning frameworksInformation and Software Technology, 2022
- Software engineering for quantum programming: How far are we?Journal of Systems and Software, 2022
- Simple mitigation of global depolarizing errors in quantum simulationsPhysical Review E, 2021
- Quantum programming languagesNature Reviews Physics, 2020
- A hybrid model of fuzzy ARTMAP and genetic algorithm for data classification and rule extractionExpert Systems with Applications, 2016
- Automated bug assignment: Ensemble-based machine learning in large scale industrial contextsEmpirical Software Engineering, 2015
- Toward an understanding of bug fix patternsEmpirical Software Engineering, 2008
- Bugzilla, ITracker, and other bug trackersIEEE Software, 2005
- Fault detection in Rule-based Software systemsInformation and Software Technology, 2003