LLM-based Dynamic Differential Testing for Database Connectors with Reinforcement Learning-Guided Prompt Selection
Authors:
Preprint
Abstract
Database connectors are critical components enabling applications to interact with underlying database management systems (DBMS), yet their security vulnerabilities often remain overlooked. Unlike traditional software defects, connector vulnerabilities exhibit subtle behavioral patterns and are inherently challenging to detect. Besides, nonstandardized implementation of connectors leaves potential risks (a.k.a. unsafe implementations) but is more elusive. As a result, traditional fuzzing methods are incapable of finding such vulnerabilities. Even for LLM-enable test case generation, due to a lack of domain knowledge, they are also incapable of generating test cases that invoke all interface and internal logic of connectors. In this paper, we propose reinforcement learning (RL)-guided LLM test-case generation for database connector testing. Specifically, to equip the LLM with sufficient and appropriate domain knowledge, a parameterized prompt template is composed which can be utilized to generate numerous prompts. Test cases are generated via LLM with a prompt, and are dynamically evaluated through differential testing across multiple connectors. The testing is iteratively conducted, with each round RL is adopted to select optimal prompt based on prior-round behavioral feedback, so as to maximize control flow coverage. We implement aforementioned methodology in a practical tool and evaluate it on two widely used JDBC connectors: MySQL Connector/J and OceanBase Connector/J. In total, we reported 16 bugs, among them 10 are officially confirmed and the rest are acknowledged as unsafe implementations.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- A Survey on Large Language Models for Code GenerationACM Transactions on Software Engineering and Methodology, 2026
- Coni: Detecting Database Connector Bugs via State-Aware Test Case Generation2025
- Understanding and Detecting SQL Function Bugs: Using Simple Boundary Arguments to Trigger Hundreds of DBMS Bugs2025
- Large Language Models for Software Engineering: A Systematic Literature ReviewACM Transactions on Software Engineering and Methodology, 2024
- A Survey of Testing Techniques Based on Large Language Models2024
- Software Testing With Large Language Models: Survey, Landscape, and VisionIEEE Transactions on Software Engineering, 2024
- Griffin : Grammar-Free DBMS Fuzzing2022
- SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback2020
- Finite-time analysis of the multi-armed bandit problem with known trend2016