Predicting the Performance-Cost Trade-off of Applications Across Multiple Systems
Authors:
- 1 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 216-228
Abstract
In modern computing environments, users may have multiple systems accessible to them such as local clusters, private clouds, or public clouds. This abundance of choices makes it difficult for users to select the system and configuration for running an application that best meet their performance and cost objectives. To assist such users, we propose a prediction tool that predicts the full performance-cost trade-off space of an application across multiple systems. Our tool runs and profiles a submitted application on a small number of configurations from some of the systems, and uses that information to predict the application's performance on all configurations in all systems. The prediction models are trained offline with data collected from running a large number of applications on a wide variety of configurations. Notable aspects of our tool include: providing different scopes of prediction with varying online profiling requirements, automating the selection of the small number of configurations and systems used for online profiling, performing online profiling using partial runs thereby make predictions for applications without running them to completion, employing a classifier to distinguish applications that scale well from those that scale poorly, and predicting the sensitivity of applications to interference from other users. We evaluate our tool using 69 data analytics and scientific computing benchmarks executing on three different single-node CPU systems with 8–9 configurations each and show that it can achieve low prediction error with modest profiling overhead.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Predicting Cloud Performance for HPC Applications: A User-Oriented Approach2017
- PARSEC3.0ACM SIGARCH Computer Architecture News, 2017
- XGBoost2016
- SPEC ACCEL: A Standard Application Suite for Measuring Hardware Accelerator Performance2015
- Paragon2013
- SPEC OMP2012 — An Application Benchmark Suite for Parallel Systems Using OpenMP2012
- No one (cluster) size fits all2011
- An Analysis of Performance Interference Effects in Virtual Environments2007
- GROMACS: Fast, flexible, and freeJournal of Computational Chemistry, 2005
- The NAS parallel benchmarks---summary and preliminary results1991