Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems
Authors:
Open Access
- 27 June 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 199-212
Abstract
Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue. We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.Keywords
Funding Information
- Office of Science of the U.S. Department of Energy (DE-AC05-00OR22725)
This publication has 19 references indexed in Scilit:
- Analysis and Correlation of Application I/O Performance and System-Wide I/O Activity2017
- Deep Residual Learning for Image Recognition2016
- Enabling Parallel Simulation of Large-Scale HPC Network SystemsIEEE Transactions on Parallel and Distributed Systems, 2016
- Techniques for modeling large-scale HPC I/O workloads2015
- Automatic Generation of I/O Kernels for HPC Applications2014
- Normalized Mutual Information Feature SelectionIEEE Transactions on Neural Networks, 2009
- 24/7 Characterization of petascale I/O workloads2009
- The Design and Implementation of a Domain-Specific Language for Network Performance TestingIEEE Transactions on Parallel and Distributed Systems, 2007
- TBBTACM SIGMETRICS Performance Evaluation Review, 2005
- A general regression neural networkIEEE Transactions on Neural Networks, 1991