WPress: An Application-Driven Performance Benchmark for Cloud-Based Virtual Machines
Authors:
- Amir Hossein BorhaniABAmir Hossein Borhani
- Philipp LeitnerPLPhilipp Leitner
- Bu-Sung LeeBLBu-Sung Lee
- Xiaorong LiXLXiaorong Li
- Terence HungTHTerence Hung
- 1 September 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 101-109
Abstract
Approaching a comprehensive performance benchmark for on-line transaction processing (OLTP) applications in a cloud environment is a challenging task. Fundamental features of clouds, such as the pay-as-you-go pricing model and unknown underlying configuration of the system, are contrary to the basic assumptions of available benchmarks such as TPC-W or RUBiS. In this paper, we introduce a systematic performance benchmark approach for OLTP applications on public clouds that use virtual machines(VMs). We propose WPress benchmark, which is based on the widespread blogging software, WordPress, as a representative OLTP application and implement an open source workload generator. Furthermore, we utilize a CPU micro-benchmark to investigate CPU performance of cloud-based VMs in greater detail. Average response time and total VM cost are the performance metrics measured by WPress. We evaluate small and large instance types of three real-life cloud providers, Amazon EC2, Microsoft Azure and Rackspace cloud. Results imply that Rackspace cloud has better average response times and total VM cost on small instances. However, Microsoft Azure is preferable for large instance type.Keywords
This publication has 26 references indexed in Scilit:
- Towards Comprehensive Measurement of Consistency Guarantees for Cloud-Hosted Data Storage ServicesLecture Notes in Computer Science, 2014
- Self-adaptive and sensitivity-aware QoS modeling for the cloud2013
- CloudBench: Experiment Automation for Cloud Environments2013
- Benchmarking in the Cloud: What It Should, Can, and Cannot Be2013
- Benchmarking OLTP/web databases in the cloud2012
- Benchmarking with Your Head in the CloudLecture Notes in Computer Science, 2012
- When Free Is Not Really Free: What Does It Cost to Run a Database Workload in the Cloud?Lecture Notes in Computer Science, 2012
- Data Management in the Cloud: Promises, State-of-the-art, and Open QuestionsDatenbank-Spektrum, 2010
- Runtime measurements in the cloudProceedings of the VLDB Endowment, 2010
- Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utilityFuture Generation Computer Systems, 2009