Performance evaluation of heterogeneous cloud functions
Authors:
- 24 August 2018
- journal article
- research article
- Published by Wiley in Concurrency and Computation: Practice and Experience
- Vol. 30 (23)
Abstract
Summary: Cloud Functions, often called Function‐as‐a‐Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous due to variations in underlying hardware, runtime systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework and one based on HyperFlow. We deployed the CPU‐intensive benchmarks: Mersenne Twister and Linpack. We measured the data transfer times between cloud functions and storage, and we measured the lifetime of the runtime environment. We evaluated all the major cloud function providers: AWS Lambda, Azure Functions, Google Cloud Functions, and IBM Cloud Functions. We made our results available online and continuously updated. We report on the results of the performance evaluation, and we discuss the discovered insights into resource allocation policies.Keywords
Funding Information
- Narodowe Centrum Nauki (2016/21/B/ST6/01497)
This publication has 19 references indexed in Scilit:
- Next generation cloud computing: New trends and research directionsFuture Generation Computer Systems, 2018
- Serverless Programming (Function as a Service)2017
- Economics of Resilient Cloud Services2016
- Infrastructure Cost Comparison of Running Web Applications in the Cloud Using AWS Lambda and Monolithic and Microservice Architectures2016
- Patterns in the Chaos—A Study of Performance Variation and Predictability in Public IaaS CloudsACM Transactions on Internet Technology, 2016
- HyperFlow: A model of computation, programming approach and enactment engine for complex distributed workflowsFuture Generation Computer Systems, 2016
- Cloud storage service benchmarking: Methodologies and experimentations2014
- The application of cloud computing to scientific workflows: a study of cost and performancePhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2013
- What Are You Paying For? Performance Benchmarking for Infrastructure-as-a-Service Offerings2011
- Montage: a grid portal and software toolkit for science-grade astronomical image mosaickingInternational Journal of Computational Science and Engineering, 2009