Cross-engine query execution in federated database systems
Authors:
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
We have developed a reference implementation of the BigDAWG system: a new architecture for future Big Data applications, guided by the philosophy that “one size does not fit all”. Such applications not only call for large-scale analytics, but also for real-time streaming support, smaller analytics at interactive speeds, data visualization, and cross-storage-system queries. The importance and effectiveness of such a system has been demonstrated in a hospital application using data from an intensive care unit (ICU). In this article, we describe the implementation and evaluation of the cross-system Query Executor. In particular, we focus on cross-engine shuffle joins within the BigDAWG system, and evaluate various strategies of computing them when faced with varying degrees of data skew.Keywords
This publication has 14 references indexed in Scilit:
- BigDAWG polystore query optimization through semantic equivalences2016
- Data transformation and migration in polystores2016
- The BigDawg monitoring framework2016
- Using a Power Law distribution to describe big data2015
- A demonstration of the BigDAWG polystore systemProceedings of the VLDB Endowment, 2015
- Skew-Aware Join Optimization for Array Databases2015
- Multiparameter Intelligent Monitoring in Intensive Care II: A public-access intensive care unit database*Critical Care Medicine, 2011
- Data placement in shared-nothing parallel database systemsThe VLDB Journal, 1997
- The POSTGRES next generation database management systemCommunications of the ACM, 1991
- Future paths for integer programming and links to artificial intelligenceComputers & Operations Research, 1986