Mixed precision algorithms in numerical linear algebra
Top Cited Papers
Open Access
- 1 May 2022
- journal article
- research article
- Published by Cambridge University Press (CUP) in Acta Numerica
- Vol. 31, 347-414
Abstract
Today’s floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is supported in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm. This survey treats a broad range of mixed precision algorithms in numerical linear algebra, both direct and iterative, for problems including matrix multiplication, matrix factorization, linear systems, least squares, eigenvalue decomposition and singular value decomposition. We identify key algorithmic ideas, such as iterative refinement, adapting the precision to the data, and exploiting mixed precision block fused multiply–add operations. We also describe the possible performance benefits and explain what is known about the numerical stability of the algorithms. This survey should be useful to a wide community of researchers and practitioners who wish to develop or benefit from mixed precision numerical linear algebra algorithms.Keywords
This publication has 156 references indexed in Scilit:
- Solving lattice QCD systems of equations using mixed precision solvers on GPUsComputer Physics Communications, 2010
- Accelerating scientific computations with mixed precision algorithmsComputer Physics Communications, 2009
- MPFRACM Transactions on Mathematical Software, 2007
- Implementation of mixed precision in solving systems of linear equations on the Cell processorConcurrency and Computation: Practice and Experience, 2007
- Iterative refinement for constrained and weighted linear least squaresBIT Numerical Mathematics, 1994
- Iterative refinement enhances the stability ofQR factorization methods for solving linear equationsBIT Numerical Mathematics, 1991
- Block elimination with one refinement solves bordered linear systems accuratelyBIT Numerical Mathematics, 1990
- Binary cascades iterative refinement in doubled-mantissa arithmeticsBIT Numerical Mathematics, 1984
- Iterative refinement implies numerical stabilityBIT Numerical Mathematics, 1977
- Iterative refinement of linear least squares solutions IBIT Numerical Mathematics, 1967