DroneTest-Copilot: LLM-Powered Automated Detection and Repair of Flaws in Drone Autotest Suites
Open Access
- 13 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network Science and Engineering
- Vol. 13, 6685-6705
Abstract
In the landscape of the low-altitude economy, ensuring the reliability of drone flight control software is of utmost importance. ArduPilot's autotest is an automated testing framework integrated into its continuous integration (CI) pipeline, which is crucial for validating autopilot functionality. However, numerous logic defects exist in its test cases, where the intended workflow doesn't match the code implementation. This paper presents DroneTest-Copilot (DTC), a novel framework that leverages large language models (LLMs) to automate the validation and correction of autotest test cases. The system consists of four components: Preprocessor, Test Case Processor, Flaw Corrector, and Vulnerability Tester. It extracts relevant context from pull requests, uses LLMs to detect flaws such as parameter mismatch, API misuse, execution order errors, and logic errors, and then generates corrected test code. We also conduct a detailed survey and comparative analysis of existing related works to highlight the novelty and positioning of DTC. Evaluations on a dataset of patched and new test case pull requests show that DTC can correctly identify 87.23% of introduced defects in the verification set and discover previously unknown vulnerabilities in the new test case set. DTC substantially reduces manual effort in maintaining autotest suites and uncovers subtle flaws in test case code, enhancing the reliability of drone flight control software.Keywords
Funding Information
- National Natural Science Foundation of China (62372191)
- National Natural Science Foundation of China (62302183)
- Open Topics from The Lion Rock Labs of Cyberspace Security
- Songshan Laboratory
- Open Foundation of Key Laboratory of Cyberspace Security, Ministry of Education of China (KLCS20240401)
- Natural Science Foundation of Hubei Province (2023AFB258)
- CCF-DiDi GAIA Collaborative Research Funds
This publication has 121 references indexed in Scilit:
- What is wrong with your code generated by large language models? An extensive studyScience China Information Sciences, 2026
- Blockchain-Based Logging: Ensuring Integrity, Immutability, and Security in Modern Systems2025
- LLM-Based Event Abstraction and Integration for IoT-Sourced Logs2025
- Automated system-level testing of unmanned aerial systemsAutomated Software Engineering, 2024
- Benchmarking Large Language Models for Log Analysis, Security, and InterpretationJournal of Network and Systems Management, 2024
- An automated model‐based testing approach for the self‐adaptive behavior of the unmanned aircraft system application softwareSoftware: Practice and Experience, 2024
- A survey on large language model based autonomous agentsFrontiers of Computer Science, 2024
- Systematic Evaluation of Different Approaches on Embedding Search2024
- Co-simulation of a Model Predictive Control System for Automotive Applications2022
- Rule-based detection of design patterns in program codeInternational Journal on Software Tools for Technology Transfer, 2013