Mutation Testing of Quantum Programs: A Case Study With Qiskit
Open Access
- 29 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Quantum Engineering
- Vol. 3, 1-17
Abstract
As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this article, building on the definition of syntactically equivalent quantum operations, we propose a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further propose QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy's performance, we conducted a case study on 24 real quantum programs written in IBM's Qiskit library. Furthermore, we show how better test suite coverage and improvements to test assertions can increase the test suites' mutation score and quality. QMutPy has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests and how to improve them.Keywords
Funding Information
- FCT/MCTES (PTDC/CCI-COM/29300/2017, CMU/TIC/0064/2019)
- research units LASIGE (UIDB/00408/2020, UIDP/00408/2020)
- INESC-ID (UIDB/50021/2020)
- LIACC (UIDB/00027/2020)
This publication has 40 references indexed in Scilit:
- Effective Regression Test Case SelectionACM Computing Surveys, 2017
- Are mutants a valid substitute for real faults in software testing?2014
- Achieving scalable model-based testing through test case diversityACM Transactions on Software Engineering and Methodology, 2013
- Experimentation in Software Engineering2012
- An Analysis and Survey of the Development of Mutation TestingIEEE Transactions on Software Engineering, 2010
- Quantum Algorithm for Linear Systems of EquationsPhysical Review Letters, 2009
- Feedback-Directed Random Test Generation29th International Conference on Software Engineering (ICSE'07), 2007
- Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum ComputerSIAM Review, 1999
- Quantum computingReports on Progress in Physics, 1998
- A fast quantum mechanical algorithm for database search1996