BeFaaS: An Application-Centric Benchmarking Framework for FaaS Platforms
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-8
Abstract
Following the increasing interest and adoption of FaaS systems, benchmarking frameworks for determining nonfunctional properties have also emerged. While existing (microbenchmark) frameworks only evaluate single aspects of FaaS platforms, a more holistic, application-driven approach is still missing. In this paper, we design and present BeFaaS, an extensible application-centric benchmarking framework for FaaS environments that focuses on the evaluation of FaaS platforms through realistic and typical examples of FaaS applications. BeFaaS includes a built-in e-commerce benchmark, is extensible for new workload profiles and new platforms, supports federated benchmark runs in which the benchmark application is distributed over multiple providers, and supports a fine-grained result analysis. Our evaluation compares three major FaaS providers in single cloud provider setups and shows that BeFaaS is capable of running each benchmark automatically with minimal configuration effort and providing detailed insights for each interaction.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Benchmarking Heterogeneous Cloud Functions2018
- The serverless trilemma: function composition for serverless computing2017
- BenchFoundry: A Benchmarking Framework for Cloud Storage Services2017
- Quality of Cloud Services: Expect the UnexpectedIEEE Internet Computing, 2017
- Patterns in the Chaos—A Study of Performance Variation and Predictability in Public IaaS CloudsACM Transactions on Internet Technology, 2016
- Towards an Extensible Middleware for Database Benchmarking2015
- WPress: An Application-Driven Performance Benchmark for Cloud-Based Virtual Machines2014
- OLTP-BenchProceedings of the VLDB Endowment, 2013
- Benchmarking in the Cloud: What It Should, Can, and Cannot Be2013
- The Art of Building a Good BenchmarkLecture Notes in Computer Science, 2009