Global Experiences with HPC Operational Data Measurement, Collection and Analysis
Authors:
- 1 September 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 499-508
Abstract
As we move into the exascale era, supercomputers grow larger, denser, more heterogeneous, and ever more complex. Operating such machines reliably and efficiently requires deep insight into the operational parameters of the machine itself as well as its supporting infrastructure. To fulfill this need, early adopter sites have started the development and deployment of Operational Data Analytics (ODA) frameworks allowing the continuous monitoring, archiving, and analysis of near realtime performance data from the machine and infrastructure levels, providing immediately actionable information for multiple operational uses. To understand their ODA goals, requirements, and use cases, we have conducted a survey among eight early adopter sites from the US, Europe, and Japan that operate top 50 high-performance computing systems. We have assessed the technologies leveraged to build their ODA frameworks, identified use cases and other push and pull factors that drive the sites' ODA activities, and report on their operational lessons.Keywords
This publication has 16 references indexed in Scilit:
- Using Machine Learning for Data Center Cooling Infrastructure Efficiency Prediction2017
- Continuous learning of HPC infrastructure models using big data analytics and in-memory processing tools2017
- Operational analytics data management systemsProceedings of the VLDB Endowment, 2016
- Supercomputing Centers and Electricity Service Providers: A Geographically Distributed Perspective on Demand Management in Europe and the United States2016
- Integrated Energy-Aware Management of Supercomputer Hybrid Cooling SystemsIEEE Transactions on Industrial Informatics, 2016
- Enterprise Architecture Intelligence: Combining Enterprise Architecture and Operational Data2014
- The 4 Pillar Framework for energy efficient HPC data centersComputer Science - Research and Development, 2013
- Matching of Process Data and Operational Data for a Deep Business Analysis2008
- Power provisioning for a warehouse-sized computer2007
- Data Mining in Manufacturing: A ReviewJournal of Manufacturing Science and Engineering, 2005