Characterizing and Subsetting Big Data Workloads
Preprint
Abstract
Big data benchmark suites must include a diversity of data and workloads to be useful in fairly evaluating big data systems and architectures. However, using truly comprehensive benchmarks poses great challenges for the architecture community. First, we need to thoroughly understand the behaviors of a variety of workloads. Second, our usual simulation-based research methods become prohibitively expensive for big data. As big data is an emerging field, more and more software stacks are being proposed to facilitate the development of big data applications, which aggravates hese challenges. In this paper, we first use Principle Component Analysis (PCA) to identify the most important characteristics from 45 metrics to characterize big data workloads from BigDataBench, a comprehensive big data benchmark suite. Second, we apply a clustering technique to the principle components obtained from the PCA to investigate the similarity among big data workloads, and we verify the importance of including different software stacks for big data benchmarking. Third, we select seven representative big data workloads by removing redundant ones and release the BigDataBench simulation version, which is publicly available from http://prof.ict.ac.cn/BigDataBench/simulatorversion/.Keywords
All Related Versions
- Version 1, 2014-09-01, ArXiv
- Published version: , 191.
This publication has 6 references indexed in Scilit:
- CloudRank-D: benchmarking and ranking cloud computing systems for data processing applicationsFrontiers of Computer Science, 2012
- Cost-Aware Cooperative Resource Provisioning for Heterogeneous Workloads in Data CentersIEEE Transactions on Computers, 2012
- In Cloud, Can Scientific Communities Benefit from the Economies of Scale?IEEE Transactions on Parallel and Distributed Systems, 2011
- Transformer: A New Paradigm for Building Data-Parallel Programming ModelsIEEE Micro, 2010
- The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale MachinesSynthesis Lectures on Computer Architecture, 2009
- Subsetting the SPEC CPU2006 benchmark suiteACM SIGARCH Computer Architecture News, 2007