SnuHPL
Authors:
Open Access
- 28 June 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Abstract
These days, it is typical for a large-scale cluster system to have different kinds of GPUs. However, HPL (High-Performance LINPACK), the de-facto standard LINPACK implementation for evaluating the performance of a cluster system, is originally designed to work only for homogeneous CPU-only systems. In this paper, we develop SnuHPL, an optimized HPL for clusters of modern heterogeneous GPUs. To optimize SnuHPL for the heterogeneous GPUs, we design a performance model, a SnuHPL simulator based on the model, and a greedy heuristic algorithm based on the simulator. The algorithm generates the best data distribution for a given cluster configuration by considering computing power, memory capacity, and network performance altogether. We also present a simple technique to increase the energy efficiency of HPL by adjusting the core clock frequency of the GPUs. The evaluation of the data distribution algorithm on small clusters of different GPU combinations shows that it outperforms well-known other data distribution strategies. We show the effectiveness of SnuHPL on a cluster of 1,760 NVIDIA A100-80GB GPUs and 440 A100-40GB GPUs. We also show the effectiveness of the proposed energy optimization technique on a cluster of 144 A100-80GB GPUs.Keywords
Funding Information
- Institute for Information and Communications Technology Promotion (2018-0-00581)
- National Research Foundation of Korea (NRF-2016M3C4A7952587, NRF-2019M3E4A1080386)
- Seoul National University (4199990214639)
This publication has 19 references indexed in Scilit:
- A Flexible and Portable Large-Scale DGEMM Library for Linpack on Next-Generation Multi-GPU Systems2015
- Accelerating LINPACK with MPI-OpenCL on Clusters of Multi-GPU NodesIEEE Transactions on Parallel and Distributed Systems, 2014
- Optimized HPL for AMD GPU and multi-core CPU usageComputer Science - Research and Development, 2011
- Computation-Communication Overlap of Linpack on a GPU-Accelerated PC ClusterIEICE Transactions on Information and Systems, 2011
- Petascale computing with acceleratorsACM SIGPLAN Notices, 2009
- An Improved Model for Predicting HPL Performance2007
- VSIPL: an object-based open standard API for vector, signal, and image processing2002
- An updated set of basic linear algebra subprograms (BLAS)ACM Transactions on Mathematical Software, 2002
- Design and Implementation of the ScaLAPACK LU, QR, and Cholesky Factorization RoutinesScientific Programming, 1995
- Performance of various computers using standard linear equations softwareACM SIGARCH Computer Architecture News, 1990