Quality-Driven Continuous Query Execution over Out-of-Order Data Streams
Authors:
Open Access
- 27 May 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 889-894
Abstract
Executing continuous queries over out-of-order data streams, where tuples are not ordered according to timestamps, is challenging; because high result accuracy and low result latency are two conflicting performance metrics. Although many applications allow trading exact query results for lower latency, they still expect the produced results to meet a certain quality requirement. However, none of existing disorder handling approaches have considered minimizing the result latency while meeting user-specified requirements on the quality of query results. In this demonstration, we showcase AQ-K-slack, an adaptive, buffer-based disorder handling approach, which supports executing sliding window aggregate queries over out-of-order data streams in a quality-driven manner. By adapting techniques from the field of sampling-based approximate query processing and control theory, AQ-K-slack dynamically adjusts the input buffer size at query runtime to minimize the result latency, while respecting a user-specified threshold on relative errors in produced query results. We demonstrate a prototype stream processing system, which extends SAP Event Stream Processor with the implementation of AQ-K-slack. Through an interactive interface, the audience will learn the effect of different factors, such as the aggregate function, the window specification, the result error threshold, and stream properties, on the latency and the accuracy of query results. Moreover, they can experience the effectiveness of AQ-K-slack in obtaining user-desired latency vs. result accuracy trade-offs, compared to naive disorder handling approaches that make extreme trade-offs. For instance, by scarifying 1% result accuracy, our system can reduce the result latency by 80% when compared to the state of the art.Keywords
This publication has 31 references indexed in Scilit:
- BlinkDB2013
- An overview of business intelligence technologyCommunications of the ACM, 2011
- Space-efficient online approximation of time series data: Streams, amnesia, and out-of-order2010
- Speculative out-of-order event processing with software transaction memory2008
- Time-decaying aggregates in out-of-order streams2008
- A Deterministic Algorithm for Summarizing Asynchronous Streams over a Sliding Window2007
- Load shedding for aggregation queries over data streams2004
- Exploiting k -constraints to reduce memory overhead in continuous queries over data streamsACM Transactions on Database Systems, 2004
- Aurora: a new model and architecture for data stream managementThe VLDB Journal, 2003
- Gigascope2002