MiniFloats on RISC-V Cores: ISA Extensions With Mixed-Precision Short Dot Products
- 19 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computing
- Vol. 12 (4) , 1040-1055
Abstract
Low-precision floating-point (FP) formats have recently been intensely investigated in the context of machine learning inference and training applications. While 16-bit formats are already widely used, 8-bit FP data types have lately emerged as a viable option for neural network training when employed in a mixed-precision scenario and combined with rounding methods increasing the precision in compound additions, such as stochastic rounding. So far, hardware implementations supporting FP8 are mostly implemented within domain-specific accelerators. We propose two RISC-V instruction set architecture (ISA) extensions, enhancing respectively scalar and vector general-purpose cores with low and mixed-precision capabilities. The extensions support two 8-bit and two 16-bit FP formats and are based on dot-product instructions accumulating at higher precision. We develop a hardware unit supporting mixed-precision dot products and stochastic rounding and integrate it into an open-source floating-point unit (FPU). Finally, we integrate the enhanced FPU into a cluster of scalar cores, as well as a cluster of vector cores, and implement them in a 12 nm FinFET technology. The former achieves 575 GFLOPS/W on FP8-to-FP16 matrix multiplications at 0.8 V, 1.26 GHz; the latter reaches 860 GFLOPS/W at 0.8 V, 1.08 GHz, 1.93x higher efficiency than computing on FP16-to-FP32.Keywords
Funding Information
- The European Pilot (101034126)
- European High Performance Computing Joint Undertaking
- FRACTAL (877056)
- Electronic Components and Systems for European Leadership
- EU Horizon 2020 research and innovation programme
This publication has 35 references indexed in Scilit:
- Mixed precision algorithms in numerical linear algebraActa Numerica, 2022
- Stochastic rounding: implementation, error analysis and applicationsRoyal Society Open Science, 2022
- Binary neural networks: A surveyPattern Recognition, 2020
- Intel Nervana Neural Network Processor-T (NNP-T) Fused Floating Point Many-Term Dot Product2020
- Variable Precision 16-Bit Floating-Point Vector Unit for Embedded Processors2020
- DLFloat: A 16-b Floating Point Format Designed for Deep Learning Training and Inference2019
- Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations2019
- Modified Fused Multiply and Add for Exact Low Precision Product Accumulation2017
- Improved Architectures for a Floating-Point Fused Dot Product Unit2013
- Handbook of Floating-Point Arithmetic2010