Jianjun Zhao

Oracle quantum programs are a fundamental class of quantum programs that enable developers to embed classical logic into quantum computing. Many important quantum algorithms are built upon oracle quantum programs, making it essential to ensure their correctness during development. Although software testing is a well-established approach for improving program reliability, no systematic method has been developed to test oracle quantum programs. This paper aims to build an equivalence-class-partition-based testing framework to complete the testing tasks for general oracle quantum programs. Our research methods involve both theoretical analysis and experimental evaluations. In the theoretical analysis, we first give the formal definition of general oracle quantum programs. Then we establish the foundational theory for the testing approaches. We propose an equivalence-class-partition-based testing framework, involving 4 testing criteria and 3 propositions about the selection of parameters in testing execution. In experimental evaluations, we select 10 original oracle quantum programs and their 132 mutants to evaluate the correctness of our theory and the effectiveness of our testing framework. Our results demonstrate that the proposed framework significantly aids developers in testing oracle quantum programs, providing insights to enhance the reliability of quantum software.

Testing quantum programs in a black-box setting is difficult because bug-relevant behavioral differences may be missed at two levels: the chosen inputs may fail to trigger faulty behavior, and even when faults are triggered, the chosen measurements may fail to make the resulting differences observable. Existing methods have mainly strengthened either input generation or measurement design in isolation, leaving their coupling insufficiently addressed. In this paper, we present QDoctor, a probe-based black-box testing framework that wraps the program under test with a variety of property-based probes, thereby effectively extracting program information and exploring its behavior. A probe integrates two components: test-case generation and the specification of the measurement basis. To reduce false positive rate, QDoctor also supports semantic-aware partial equivalence testing through constrained probe-oracle design. Experimental results on benchmark suites show that QDoctor achieves the best balanced performance among representative black-box baselines, reaching 99.89% recall, 98.23% precision, 99.06% F1, and 0.9988 AUC. Ablation shows that jointly strengthening inputs and measurements raises recall from 85.280% to 99.728% with zero false positives, while semantic probes drive the false-positive rate to zero on SWAP-induced equivalent mutants.

Large Language Models (LLMs) have already reshaped many aspects of traditional software engineering, from code generation and debugging to automated testing and maintenance. As an important research sub-domain in the software engineering area, quantum software engineering (QSE) is poised to undergo a similar transformation. As QSE moves toward broader adoption, including the generation of quantum code, program testing, and automated program repair, LLMs will likely play a key supporting role in its development. However, despite encouraging early attempts, the community still lacks a roadmap for using LLMs in QSE. Without such a roadmap, opportunities may be missed, challenges may remain unrevealed, and risks may be overlooked. In this article, we present our vision for how LLMs can advance QSE, focusing on four core tasks: program modeling and analysis, program generation, testing, and repair. We outline the opportunities and unique challenges, identify the risks inherent in this integration, and propose a research agenda to guide the community toward realizing the full potential of LLM-powered QSE.

Formal verification of transformers has become increasingly important due to their widespread deployment in safety-critical applications. Compared to classic neural networks, the inferences of transformers involve highly complex computations, such as dot products in self-attention layers, rendering their verification extremely difficult. Existing approaches explored over-approximation methods by constructing convex constraints to bound the output ranges of transformers, which can achieve high efficiency. However, they may sacrifice verification precision, and consequently introduce significant approximation error that leads to frequent occurrences of false alarms. In this paper, we propose a transformer verification approach that can achieve improved precision. At the core of our approach is a novel usage of ReLU, by which we represent a precise but non-linear bound for dot products such that we can further exploit the rich body of literature for convex relaxation of ReLU to derive precise bounds. We extend two classic approaches to the context of transformers, a rule-based one and an optimization-based one, resulting in two new frameworks for efficient and precise verification. We evaluate our approaches on different model architectures and robustness properties derived from two datasets about sentiment analysis, and compare with the state-of-the-art baseline approach. Compared to the baseline, our approach can achieve significant precision improvement for most of the verification tasks with acceptable compromise of efficiency, which demonstrates the effectiveness of our approach.

With the growing synergy between deep learning and quantum computing, Quantum Neural Networks (QNNs) have emerged as a promising paradigm by leveraging quantum parallelism and entanglement. However, testing QNNs remains underexplored due to their complex quantum dynamics and limited interpretability. Developing a mutation testing technique for QNNs is promising while requires addressing stochastic factors, including the inherent randomness of mutation operators and quantum measurements. To tackle these challenges, we propose QuanForge, a mutation testing framework specifically designed for QNNs. We first introduce statistical mutation killing to provide a more reliable criterion. QuanForge incorporates nine post-training mutation operators at both gate and parameter levels, capable of simulating various potential errors in quantum circuits. Finally, a mutant generation algorithm is formalized that systematically produces effective mutants, thereby enabling a robust and reliable mutation analysis. Through extensive experiments on benchmark datasets and QNN architectures, we show that QuanForge can effectively distinguish different test suites and localize vulnerable circuit regions, providing insights for data enhancement and structural assessment of QNNs. We also analyze the generation capabilities of different operators and evaluate performance under simulated noisy conditions to assess the practical feasibility of QuanForge for future quantum devices.

In quantum software engineering (QSE), quantum software testing (QST) has attracted increasing attention as quantum software systems grow in scale and complexity. Since QST evaluates quantum programs through execution under designed test inputs, empirical studies are widely used to assess the effectiveness of testing approaches. However, the design and reporting of empirical studies in QST remain highly diverse, and a shared methodological understanding has yet to emerge, making it difficult to interpret results and compare findings across studies. To address this, this paper presents a methodological analysis of empirical studies in QST through a systematic examination of 59 primary studies identified from a literature pool of size 384. We organize our analysis around ten research questions that cover key methodological dimensions of QST empirical studies, including objects under test, baseline comparison, testing setup, experimental configuration, and tool and artifact support. Through cross-study analysis along these dimensions, we characterize current empirical practices in QST, identify recurring limitations and inconsistencies, and highlight open methodological challenges. Based on our findings, we derive insights and recommendations to inform the design, execution, and reporting of future empirical studies in QST.

With the growing interest in quantum programs, ensuring their correctness is a fundamental challenge. Although constraint-solving techniques can overcome some limitations of traditional testing and verification, they have not yet been sufficiently explored in the context of quantum programs. To address this gap, we present QSolver, the first quantum constraint solver. QSolver provides a structured framework for handling five types of quantum constraints and incorporates an automated assertion generation module to verify quantum states. QSolver transforms quantum programs and multi-moment constraints into symbolic representations, and utilizes an SMT solver to obtain quantum states that satisfy these constraints. To validate the correctness of the generated input states, QSolver automatically generates assertion programs corresponding to each constraint. Experimental results show that QSolver efficiently processes commonly used quantum gates and demonstrates good scalability across quantum programs of different sizes.