Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing
Authors:
Preprint
Abstract
Language models have become practical tools for quantum computing education and research, from summarizing technical papers to explaining theoretical concepts and answering questions about recent developments in the field. While existing benchmarks evaluate quantum code generation and circuit design, their understanding of quantum computing concepts has not been systematically measured. Quantum-Audit addresses this gap with 2,700 questions covering core quantum computing topics. We evaluate 26 models from leading organizations. Our benchmark comprises 1,000 expert-written questions, 1,000 questions extracted from research papers using LLMs and validated by experts, plus an additional 700 questions including 350 open-ended questions and 350 questions with false premises to test whether models can correct erroneous assumptions. Human participants scored between 23% and 86%, with experts averaging 74%. Top-performing models exceeded the expert average, with Claude Opus 4.5 reaching 84% accuracy, though top models showed an average 12-point accuracy drop on expert-written questions compared to LLM-generated ones. Performance declined further on advanced topics, dropping to 73% on security questions. Additionally, models frequently accepted and reinforced false premises embedded in questions instead of identifying them, with accuracy below 66% on these critical reasoning tasks.Keywords
All Related Versions
This publication has 81 references indexed in Scilit:
- Differentiable architecture search for adversarially robust quantum computer visionQuantum Machine Intelligence, 2026
- IBM quantum computers: evolution, performance, and future directionsThe Journal of Supercomputing, 2025
- Quantum resource estimation for large scale quantum algorithmsFuture Generation Computer Systems, 2024
- Securing IoT devices: A novel approach using blockchain and quantum cryptographyInternet of Things, 2023
- The state of quantum computing applications in health and medicineResearch Directions: Quantum Technologies, 2023
- Time complexity analysis of quantum algorithms via linear representations for nonlinear ordinary and partial differential equationsJournal of Computational Physics, 2023
- Supervised Learning with Quantum Computers2018
- Path selection for quantum repeater networksNetworking Science, 2013
- A scheme for efficient quantum computation with linear opticsNature, 2001
- The computer as a physical system: A microscopic quantum mechanical Hamiltonian model of computers as represented by Turing machinesJournal of Statistical Physics, 1980