Mixed-precision pre-pivoting strategy for the LU factorization
Open Access
- 28 October 2024
- journal article
- research article
- Published by Springer Nature in The Journal of Supercomputing
- Vol. 81 (1) , 1-46
Abstract
This paper investigates the efficient application of half-precision floating-point (FP16) arithmetic on GPUs for boosting LU decompositions in double (FP64) precision. Addressing the motivation to enhance computational efficiency, we introduce two novel algorithms: Pre-Pivoted LU (PRP) and Mixed-precision Panel Factorization (MPF). Deployed in both hybrid CPU-GPU setups and native GPU-only configurations, PRP identifies pivot lists through LU decomposition computed in reduced precision and subsequently reorders matrix rows in FP64 precision before executing LU decomposition without pivoting. Two variants of PRP, namely hPRP and xPRP, are introduced, differing in their computation of pivot lists in full half-precision or mixed half-single precision. The MPF algorithm generates FP64 LU factorization while internally utilizing hPRP for panel factorization, showcasing accuracy on par with standard DGETRF but with superior speed. The study further explores auxiliary functions required for the native mode implementation of PRP variants and MPF.Keywords
All Related Versions
Funding Information
- Agencia Estatal de Investigación (RYC2021-033973-I)
- Universitat Jaume I (UJI-2023-04)
- Generalitat de Catalunya (2021-SGR-01007)
- Ministerio de Economía y Competitividad (PID2019-107255GB)
- Universitat Jaume I
This publication has 26 references indexed in Scilit:
- Average-case analysis of the Gaussian elimination with partial pivotingProbability Theory and Related Fields, 2024
- Accelerating Variants of the Conjugate Gradient with the Variable Precision Processor2022
- Investigating the Benefit of FP16-Enabled Mixed-Precision Solvers for Symmetric Positive Definite Matrices Using GPUs2020
- Numerical algorithms for high-performance computational sciencePhilosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2020
- New Flexible Multiple-Precision Multiply-Accumulate Unit for Deep Neural Network Training and InferenceIEEE Transactions on Computers, 2019
- IEEE Standard for Floating-Point Arithmetic2019
- Harnessing GPU Tensor Cores for Fast FP16 Arithmetic to Speed up Mixed-Precision Iterative Refinement Solvers2018
- Multiple Precision Floating-Point Arithmetic on SIMD Processors2017
- IEEE Standard for Floating-Point Arithmetic2008
- A geometric analysis of Gaussian elimination. ILinear Algebra and its Applications, 1991