A factor framework for experimental design for performance evaluation of commercial cloud services
Authors:
- 1 December 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 169-176
Abstract
Given the diversity of commercial Cloud services, performance evaluations of candidate services would be crucial and beneficial for both service customers (e.g. cost-benefit analysis) and providers (e.g. direction of service improvement). Before an evaluation implementation, the selection of suitable factors (also called parameters or variables) plays a prerequisite role in designing evaluation experiments. However, there seems a lack of systematic approaches to factor selection for Cloud services performance evaluation. In other words, evaluators randomly and intuitively concerned experimental factors in most of the existing evaluation studies. Based on our previous taxonomy and modeling work, this paper proposes a factor framework for experimental design for performance evaluation of commercial Cloud services. This framework capsules the state-of-the-practice of performance evaluation factors that people currently take into account in the Cloud Computing domain, and in turn can help facilitate designing new experiments for evaluating Cloud services.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- On a Catalogue of Metrics for Evaluating Commercial Cloud Services2012
- Performance Analysis of Cloud Computing Services for Many-Tasks Scientific ComputingIEEE Transactions on Parallel and Distributed Systems, 2011
- Performance Evaluation of Computer and Communication Systems2011
- CloudCmp2010
- Abstractions for Loosely-Coupled and Ensemble-Based Simulations on Azure2010
- Early observations on the performance of Windows Azure2010
- Case study for running HPC applications in public clouds2010
- Scientific workflow applications on Amazon EC22009
- A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster?2009
- Model-driven development - Guest editor's introductionIEEE Software, 2003