High-performance bidiagonal reduction using tile algorithms on homogeneous multicore architectures
Authors:
Open Access
- 1 April 2013
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Mathematical Software
- Vol. 39 (3) , 1-22
Abstract
This article presents a new high-performance bidiagonal reduction (BRD) for homogeneous multicore architectures. This article is an extension of the high-performance tridiagonal reduction implemented by the same authors [Luszczek et al., IPDPS 2011] to the BRD case. The BRD is the first step toward computing the singular value decomposition of a matrix, which is one of the most important algorithms in numerical linear algebra due to its broad impact in computational science. The high performance of the BRD described in this article comes from the combination of four important features: (1) tile algorithms with tile data layout, which provide an efficient data representation in main memory; (2) a two-stage reduction approach that allows to cast most of the computation during the first stage (reduction to band form) into calls to Level 3 BLAS and reduces the memory traffic during the second stage (reduction from band to bidiagonal form) by using high-performance kernels optimized for cache reuse; (3) a data dependence translation layer that maps the general algorithm with column-major data layout into the tile data layout; and (4) a dynamic runtime system that efficiently schedules the newly implemented kernels across the processing units and ensures that the data dependencies are not violated. A detailed analysis is provided to understand the critical impact of the tile size on the total execution time, which also corresponds to the matrix bandwidth size after the reduction of the first stage. The performance results show a significant improvement over currently established alternatives. The new high-performance BRD achieves up to a 30-fold speedup on a 16-core Intel Xeon machine with a 12000× 12000 matrix size against the state-of-the-art open source and commercial numerical software packages, namely LAPACK, compiled with optimized and multithreaded BLAS from MKL as well as Intel MKL version 10.2.Keywords
Funding Information
- National Science Foundation
- Microsoft Research
- U.S. Department of Energy
This publication has 28 references indexed in Scilit:
- Analysis of dynamically scheduled tile algorithms for dense linear algebra on multicore architecturesConcurrency and Computation: Practice and Experience, 2011
- A class of parallel tiled linear algebra algorithms for multicore architecturesParallel Computing, 2009
- A new stable bidiagonal reduction algorithmLinear Algebra and its Applications, 2005
- One-sided reduction to bidiagonal formLinear Algebra and its Applications, 2003
- Algorithm 807ACM Transactions on Mathematical Software, 2000
- Efficient parallel reduction to bidiagonal formParallel Computing, 1999
- A Parallel Algorithm for Computing the Singular Value Decomposition of a MatrixSIAM Journal on Matrix Analysis and Applications, 1994
- Principal component analysis in linear systems: Controllability, observability, and model reductionIEEE Transactions on Automatic Control, 1981
- Unitary Triangularization of a Nonsymmetric MatrixJournal of the ACM, 1958
- Analysis of a complex of statistical variables into principal components.Journal of Educational Psychology, 1933