High-Performance Tensor Contraction without Transposition
Author:
- Devin A. MatthewsDMDevin A. Matthews
- 1 January 2018
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Scientific Computing
- Vol. 40 (1) , C1-C24
Abstract
Tensor computations---in particular tensor contraction (TC)---are important kernels in many scientific computing applications. Due to the fundamental similarity of TC to matrix multiplication and to the availability of optimized implementations such as the BLAS, tensor operations have traditionally been implemented in terms of BLAS operations, incurring both a performance and a storage overhead. Instead, we implement TC using the flexible BLAS-like Instantiation Software (BLIS) framework, which allows for transposition (reshaping) of the tensor to be fused with internal partitioning and packing operations, requiring no explicit transposition operations or additional workspace. This implementation, TBLIS, achieves performance approaching that of matrix multiplication, and in some cases considerably higher than that of traditional TC. Our implementation supports multithreading using an approach identical to that used for matrix multiplication in BLIS, with similar performance characteristics. The complexity...Keywords
All Related Versions
Funding Information
- Intel Corporation
- National Science Foundation (ACI-1148125/1340293)
This publication has 32 references indexed in Scilit:
- Optimizing tensor contraction expressions for hybrid CPU-GPU executionCluster Computing, 2011
- Performance Optimization of Tensor Contraction Expressions for Many-Body Methods in Quantum ChemistryThe Journal of Physical Chemistry A, 2009
- High-performance implementation of the level-3 BLASACM Transactions on Mathematical Software, 2008
- Anatomy of high-performance matrix multiplicationACM Transactions on Mathematical Software, 2008
- Coupled-cluster theory in quantum chemistryReviews of Modern Physics, 2007
- Algorithm 862ACM Transactions on Mathematical Software, 2006
- Tensor Contraction Engine: Abstraction and Automated Parallel Implementation of Configuration-Interaction, Coupled-Cluster, and Many-Body Perturbation TheoriesThe Journal of Physical Chemistry A, 2003
- A set of level 3 basic linear algebra subprogramsACM Transactions on Mathematical Software, 1990
- An extended set of FORTRAN basic linear algebra subprogramsACM Transactions on Mathematical Software, 1988
- Basic Linear Algebra Subprograms for Fortran UsageACM Transactions on Mathematical Software, 1979