MIA: Metric Importance Analysis for Big Data Workload Characterization
Authors:
- 4 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 29 (6) , 1371-1384
Abstract
Data analytics is at the foundation of both high-quality products and services in modern economies and societies. Big data workloads run on complex large-scale computing clusters, which implies significant challenges for deeply understanding and characterizing overall system performance. In general, performance is affected by many factors at multiple layers in the system stack, hence it is challenging to identify the key metrics when characterizing and understanding big data workload performance. In this paper, we propose a novel workload characterization methodology using ensemble learning, called Metric Importance Analysis (MIA), to quantify the respective importance of workload metrics. By focusing on the most important metrics, MIA reduces the complexity of the analysis without losing information. Moreover, we develop the MIA-based Kiviat Plot (MKP) and Benchmark Similarity Matrix (BSM) which provide more insightful information than the traditional linkage clustering based dendrogram to visualize program behavior (dis)similarity. To demonstrate the applicability of MIA, we use it to characterize three big data benchmark suites: HiBench, CloudRank-D and SZTS. The results show that MIA is able to characterize complex big data workloads in a simple, intuitive manner, and reveal interesting insights. Moreover, through a case study, we demonstrate that tuning the configuration parameters related to the important metrics found by MIA results in higher performance improvements than through tuning the parameters related to the less important ones.Keywords
Funding Information
- National Key R&D Program of China (2016YFB1000204)
- NSFC (61672511, 61702495)
- Outstanding Technical Talent Program of CAS
- Major Scientific and Technological Project of Guangdong Province (2014B010115003)
- Shenzhen Technology Research Project (JSGG20160510154-636747)
- Key technique research on Haiyun Data System of NICT (XDA06010500)
This publication has 35 references indexed in Scilit:
- Multiple additive regression trees with application in epidemiologyStatistics in Medicine, 2003
- Stochastic gradient boostingComputational Statistics & Data Analysis, 2002
- Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 2001
- Support-vector networksMachine Learning, 1995
- Shape in ranking Kiviat graphsACM SIGMETRICS Performance Evaluation Review, 1976
- Further comments on comparative evaluation of Kiviat graphsACM SIGMETRICS Performance Evaluation Review, 1975
- Kiviat graphsACM SIGMETRICS Performance Evaluation Review, 1974
- A technique for comparative analysis of Kiviat graphsACM SIGMETRICS Performance Evaluation Review, 1974
- Software unit profiles & Kiviat figuresACM SIGMETRICS Performance Evaluation Review, 1973
- Generalized Linear ModelsJournal of the Royal Statistical Society. Series A (General), 1972