Abstract
The growing maturity of quantum computing has led to increasingly complex software ecosystems spanning frameworks, compilers, simulators, and application-level programs. Despite this rapid growth, the community still lacks a realistic benchmark for evaluating automated quantum bug fixing under quantum-specific correctness requirements, such as stochastic outputs, tolerance-based equivalence, and environment-sensitive execution. To bridge this gap, we present QBench, an end-to-end benchmark for evaluating automated repair of real-world quantum software bugs. QBench consists of two complementary tracks: RepoTrack, which targets defects in quantum software infrastructure and repositories, and ScriptTrack, which targets bugs in user-facing quantum programs and experimental workflows. Each benchmark instance provides a reproducible bug-fixing task together with the execution environment, correctness oracle, and evaluation protocol required to validate candidate repairs, enabling standardized assessment of repair systems under realistic development settings. QBench supports deterministic, tolerance-aware, and probabilistic correctness oracles, enabling reliable evaluation under realistic quantum execution semantics. Using QBench, we conduct a large-scale empirical study of automated quantum software repair and analyze the challenges faced by representative repair approaches. Our results show that stochastic correctness criteria, environment fragility, and cross-layer defects remain major obstacles to successful repair, highlighting the unique difficulties of maintaining and repairing quantum software.
Funding Information
  • Ningbo Yongjiang Talent Programme (-)
  • CPS-Yangtze Delta Region Industrial Innovation Center of Quantum and Information Technology-MindSpore Quantum Open Fund (-)