DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs
Authors:
- 28 October 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Mixed-precision neural networks (MPNNs) that enable the use of just enough data width for a deep learning task promise significant advantages of both inference accuracy and computing overhead. FPGAs with fine-grained reconfiguration capability can adapt the processing with distinct data width and models, and hence, can theoretically unleash the potential of MPNNs. Nevertheless, commodity DPUs on FPGAs mostly emphasize generality and have limited support for MPNNs especially the ones with lower data width. In addition, primitive DSPs in FPGAs usually have much larger data width than that is required by MPNNs and haven't been sufficiently co-explored with MPNNs yet. To this end, we propose an open source MPNN accelerator design framework specifically tailored for FPGAs. In this framework, we have a systematic DSP-packing algorithm to pack multiple lower data width MACs in a single primitive DSP and enable efficient implementation of MPNNs. Meanwhile, we take DSP packing efficiency into consideration with MPNN quantization within a unified neural network architecture search (NAS) framework such that it can be aware of the DSP overhead during quantization and optimize the MPNN performance and accuracy concurrently. Finally, we have the optimized MPNN fine-tuned to a fully pipelined neural network accelerator template based on HLS and make best use of available resources for higher performance. Our experiments reveal the resulting accelerators produced by the proposed framework can achieve overwhelming advantages in terms of performance, resource utilization, and inference accuracy for MPNNs when compared with both handcrafted counterparts and prior hardware-aware neural network accelerators on FPGAs.Keywords
All Related Versions
Funding Information
- National Key R&D Program of China (2022YFB4500405)
- National Natural Science Foundation of China (62174162)
This publication has 19 references indexed in Scilit:
- FTDL: A Tailored FPGA-Overlay for Deep Learning with High Scalability2020
- Rethinking Differentiable Search for Mixed-Precision Neural Networks2020
- You Only Search Once: A Fast Automation Framework for Single-Stage DNN/Accelerator Co-design2020
- Mixed Precision Neural Architecture Search for Energy Efficient Deep Learning2019
- Scaling the Cascades: Interconnect-Aware FPGA Implementation of Machine Learning Problems2019
- DAC-SDC Low Power Object Detection Challenge for UAV ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- HAQ: Hardware-Aware Automated Quantization With Mixed Precision2019
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search2019
- DNNBuilder2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018