Open Access
- 1 July 2021
- journal article
- conference paper
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 14 (11) , 2419-2431
Abstract
Data analysis often involves comparing subsets of data across many dimensions for finding unusual trends and patterns. While the comparison between subsets of data can be expressed using SQL, they tend to be complex to write, and suffer from poor performance over large and high-dimensional datasets. In this paper, we propose a new logical operator COMPARE for relational databases that concisely captures the enumeration and comparison between subsets of data and greatly simplifies the expressing of a large class of comparative queries. We extend the database engine with optimization techniques that exploit the semantics of COMPARE to significantly improve the performance of such queries. We have implemented these extensions inside Microsoft SQL Server, a commercial DBMS engine. Our extensive evaluation on synthetic and real-world datasets shows that COMPARE results in a significant speedup over existing approaches, including physical plans generated by today's database systems, user-defined functions (UDFs), as well as middleware solutions that compare subsets outside the databases.Keywords
This publication has 28 references indexed in Scilit:
- S ee DBProceedings of the VLDB Endowment, 2015
- The case for data visualization management systemsProceedings of the VLDB Endowment, 2014
- Sturges' ruleWIREs Computational Statistics, 2009
- Using grouping variables to express complex decision support queriesData & Knowledge Engineering, 2007
- Aggregate nearest neighbor queries in spatial databasesACM Transactions on Database Systems, 2005
- Efficient set joins on similarity predicates2004
- The k-Nearest Neighbour Join: Turbo Charging the KDD ProcessKnowledge and Information Systems, 2004
- Answering complex SQL queries using automatic summary tables2000
- Similarity-based queries for time series dataACM SIGMOD Record, 1997
- Providing better support for a class of decision support queriesACM SIGMOD Record, 1996