Micro-kernels for portable and efficient matrix multiplication in deep learning
Open Access
- 14 December 2022
- journal article
- research article
- Published by Springer Nature in The Journal of Supercomputing
- Vol. 79 (7) , 8124-8147
Abstract
We provide a practical demonstration that it is possible to systematically generate a variety of high-performance micro-kernels for the general matrix multiplication (gemm) via generic templates which can be easily customized to different processor architectures and micro-kernel dimensions. These generic templates employ vector intrinsics to exploit the SIMD (single instruction, multiple data) units in current general-purpose processors and, for the particular type of gemm problems encountered in deep learning, deliver a floating-point throughput rate on par with or even higher than that obtained with conventional, carefully tuned implementations of gemm in current linear algebra libraries (e.g., BLIS, AMD AOCL, ARMPL). Our work exposes the structure of the template-based micro-kernels for ARM Neon (128-bit SIMD), ARM SVE (variable-length SIMD) and Intel AVX512 (512-bit SIMD), showing considerable performance for an NVIDIA Carmel processor (ARM Neon), a Fujitsu A64FX processor (ARM SVE) and on an AMD EPYC 7282 processor (256-bit SIMD).Keywords
All Related Versions
Funding Information
- Ministerio de Ciencia e Innovación (RTI2018-093684-B-I00, FJC2019-039222-I)
- Ministerio de Ciencia e Innovación (PID2020-113656RB-C22, PID2021-126576NB-I00)
- Consejería de Transformación Económica, Industria, Conocimiento y Universidades de la Junta de Andalucía
- European Comission (PR65/19-22445)
- Comunidad de Madrid (S2018/TCS-4423)
- Universidad Politecnica de Valencia
This publication has 13 references indexed in Scilit:
- Anatomy of the BLIS Family of Algorithms for Matrix Multiplication2022
- Deep Residual Learning for Image Recognition2016
- BLIS: A Framework for Rapidly Instantiating BLAS FunctionalityACM Transactions on Mathematical Software, 2015
- Anatomy of High-Performance Many-Threaded Matrix Multiplication2014
- Model-driven Level 3 BLAS Performance Optimization on Loongson 3A Processor2012
- RooflineCommunications of the ACM, 2009
- Anatomy of high-performance matrix multiplicationACM Transactions on Mathematical Software, 2008
- Scalable parallelization of FLAME code via the workqueuing modelACM Transactions on Mathematical Software, 2008
- A Family of High-Performance Matrix Multiplication Algorithms2006
- A set of level 3 basic linear algebra subprogramsACM Transactions on Mathematical Software, 1990