Data Generation for Testing and Grading SQL Queries
Authors:
- Bikash ChandraBCBikash Chandra
- Amol Bhangdia # •A•Amol Bhangdia # •
- Bhupesh ChawdaBCBhupesh Chawda
- Biplab KarBKBiplab Kar
- K V Maheshwara Reddy # •K•K V Maheshwara Reddy # •
- Shetal ShahSSShetal Shah
- • S Sudarshan•S• S Sudarshan
- Iit BombayIBIit Bombay
Preprint
Abstract
Correctness of SQL queries is usually tested by executing the queries on one or more datasets. Erroneous queries are often the results of small changes, or mutations of the correct query. A mutation Q' of a query Q is killed by a dataset D, if Q(D) $\neq$ Q'(D). Earlier work on the XData system showed how to generate datasets that kill all mutations in a class of mutations that included join type and comparison operation mutations. In this paper we extend the XData data generation techniques to handle a wider variety of SQL queries and a much larger class of mutations. We have also built a system for grading SQL queries using the datasets generated by XData. We present a study of the effectiveness of the datasets generated by the extended XData approach, using a variety of queries including queries submitted by students as part of a database course. We show that the XData datasets outperform predefined datasets as well as manual grading done earlier by teaching assistants, while also avoiding the drudgery of manual correction. Thus, we believe that our techniques will be of great value to database course instructors and TAs, particularly to those of MOOCs. It will also be valuable to database application developers and testers for testing SQL queries.
Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- The XDa-TA system for automated grading of SQL query assignments2015
- Simple linear string constraintsFormal Aspects of Computing, 2013
- Generating test data for killing SQL mutants: A constraint-based approach2011
- Constraint-based test database generation for SQL queries2010
- Rex: Symbolic Regular Expression Explorer2010
- Full predicate coverage for testing SQL database queriesSoftware Testing, Verification and Reliability, 2010
- Qex: Symbolic SQL Query Explorer2010
- X-data: Generating test data for killing SQL mutants2010
- Symbolic Query Exploration2009
- The complexity of querying indefinite data about linearly ordered domains1992