A 3-D Multi-Precision Scalable Systolic FMA Architecture
- 19 November 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 72 (1) , 265-276
Abstract
Artificial Intelligence (AI) has almost become the default approach in a wide range of applications, such as computer vision, chatbots, and natural language processing. These AI-based applications require computing large-scale data with sufficient precision, typically in floating-point numbers, within a limited time window. A primary target for AI acceleration is matrix multiplication, mainly involving dot products through Multiply-Accumulate (MAC) operations. Current research employs the Fused Multiply-Add (FMA) operation, based on IEEE-754 Floating Point (FP) standard, to meet these requirements. However, current research focuses more on simplifying the internal digital circuits of the Processing Elements (PEs) performing FMA operations, rather than optimizing the FMA process specifically for MAC tasks. Current PE arrays often use a two-dimensional (2-D) systolic array design, without specific optimization for MAC operations, thus their parallelism is not fully utilized. Additionally, these designs lack reconfigurability and flexibility, leading to suboptimal performance on Field-Programmable Gate Arrays (FPGAs). Moreover, some designs adopt lower precision computing in AI inference for higher performance. However, some AI models still rely on high-precision computing to maintain the accuracy. Thus, multi-precision computing is commonly used in AI accelerators. To address these challenges, this paper proposes a novel Multi-Fused Multiply-Accumulate (MFMA) scheme and a corresponding three-dimensional (3-D) scalable systolic FP computing architecture. The MFMA scheme addresses the problem of the classical FMA scheme. It optimizes FMA for MAC operations with the Fused Multiply-Accumulate (FMAC) operation. Also, it combines multi-precision and mixed-precision FP computing methods for higher accuracy and lower overflow error. The proposed architecture integrates two 2-D systolic arrays into the PE for a 3-D systolic array, achieving higher parallelism and flexibility. The proposed scalable architecture can be customized to suit various FMAC operations. Compared with existing state-of-the-art FP architectures on FPGAs, our proposed architecture achieves 47%, 10%, and 159% energy efficiency improvements in FP32, FP16, and INT8 operations, respectively. Furthermore, our proposed architecture achieves energy efficiency improvements of 105%, 54%, and 262% under efficiency saturation conditions, outperforming the existing state-of-the-art design.Keywords
This publication has 35 references indexed in Scilit:
- A Case for Low Bitwidth Floating Point Arithmetic on FPGA for Transformer Based DNN Inference2024
- Performance Characterization of Using Quantization for DNN Inference on Edge Devices2023
- Efficient Device-Edge Inference for Disaster Classification2022
- FitNN: A Low-Resource FPGA-Based CNN Accelerator for DronesIEEE Internet of Things Journal, 2022
- FPGA-based Floating Point Fractional Order Image Edge Detection2019
- Area‐ and power‐efficient iterative single/double‐precision merged floating‐point multiplier on FPGAIET Computers & Digital Techniques, 2017
- A Multi-functional Multi-precision 4D Dot Product Unit with SIMD ArchitectureArabian Journal for Science and Engineering, 2016
- Design and Implementation of an Embedded FPGA Floating Point DSP Block2015
- Area-efficient architectures for double precision multiplier on FPGA, with run-time-reconfigurable dual single precision supportMicroelectronics Journal, 2013
- Accelerating scientific computations with mixed precision algorithmsComputer Physics Communications, 2009