Communication-reduced Conjugate Gradient Variants for GPU-accelerated Clusters
Authors:
Preprint
Abstract
Linear solvers are key components in any software platform for scientific and engineering computing. The solution of large and sparse linear systems lies at the core of physics-driven numerical simulations relying on partial differential equations (PDEs) and often represents a significant bottleneck in datadriven procedures, such as scientific machine learning. In this paper, we present an efficient implementation of the preconditioned s-step Conjugate Gradient (CG) method, originally proposed by Chronopoulos and Gear in 1989, for large clusters of Nvidia GPU-accelerated computing nodes. The method, often referred to as communication-reduced or communication-avoiding CG, reduces global synchronizations and data communication steps compared to the standard approach, enhancing strong and weak scalability on parallel computers. Our main contribution is the design of a parallel solver that fully exploits the aggregation of low-granularity operations inherent to the s-step CG method to leverage the high throughput of GPU accelerators. Additionally, it applies overlap between data communication and computation in the multi-GPU sparse matrix-vector product. Experiments on classic benchmark datasets, derived from the discretization of the Poisson PDE, demonstrate the potential of the method.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Mixed precisions‐step Lanczos and conjugate gradient algorithmsNumerical Linear Algebra with Applications, 2021
- Parallel performance of algebraic multigrid domain decompositionNumerical Linear Algebra with Applications, 2020
- AMG based on compatible weighted matching for GPUsParallel Computing, 2020
- SParC-LES: Enabling large eddy simulations with parallel sparse matrix computation toolsComputers & Mathematics with Applications, 2015
- A massively parallel solver for discrete Poisson-like problemsJournal of Computational Physics, 2015
- Domain Decomposition Preconditioners for Communication-Avoiding Krylov Methods on a Hybrid CPU/GPU Cluster2014
- Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithmParallel Computing, 2014
- Communication lower bounds and optimal algorithms for numerical linear algebraActa Numerica, 2014
- On the efficient implementation of preconditioned s-step conjugate gradient methods on multiprocessors with memory hierarchyParallel Computing, 1989
- Multitasking the conjugate gradient method on the CRAY X-MP/48Parallel Computing, 1987