Warper: Efficiently Adapting Learned Cardinality Estimators to Data and Workload Drifts
Authors:
Open Access
- 10 June 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1920-1933
Abstract
Recent learned cardinality estimation (CE) models are vulnerable when query predicates or the underlying datasets drift from what the models were trained upon. We propose a system Warper that accelerates model adaptation to drifts; Warper generates additional queries when limited examples are available from the new workload and carefully picks which queries to use to update the CE model. We show that Warper can be used to adapt different CE models including ones that support queries over single tables and join expressions. Experiments with different drifts suggest that Warper has a small computational cost and adapts much faster compared to state-of-the-art solutions. We also show that faster model adaptation improves query performance by shortening the period for which imperfect query plans are picked by a query optimizer due to incorrect cardinality estimates.Keywords
This publication has 31 references indexed in Scilit:
- Learning in Nonstationary Environments: A SurveyIEEE Computational Intelligence Magazine, 2015
- Recommender system application developments: A surveyDecision Support Systems, 2015
- A survey on concept drift adaptationACM Computing Surveys, 2014
- Efficiently adapting graphical models for selectivity estimationThe VLDB Journal, 2012
- Best practices for convolutional neural networks applied to visual document analysis2005
- Active learning using pre-clustering2004
- Selectivity estimation using probabilistic models2001
- An overview of query optimization in relational systems1998
- DATA AUGMENTATION AND DYNAMIC LINEAR MODELSJournal of Time Series Analysis, 1994
- Principal component analysisChemometrics and Intelligent Laboratory Systems, 1987