FitNN: A Low-Resource FPGA-Based CNN Accelerator for Drones
- 30 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 9 (21) , 21357-21369
Abstract
Executing deep neural networks (DNNs) on resource-constraint edge devices, such as drones, offers low inference latency, high data privacy, and reduced network traffic. However, deploying DNNs on such devices is a challenging task. During DNN inference, intermediate results require significant data movement and frequent off-chip memory (DRAM) access, which decreases the inference speed and power efficiency. To address this issue, this paper presents a field-programmable gate array (FPGA)-based convolutional neural network (CNN) accelerator, named FitNN, which improves the speed and power efficiency of CNN inference by reducing data movements. FitNN adopts a pre-trained CNN of iSmart2, which is composed of depthwise and pointwise blocks in the Mobilenet structure. A cross-layer dataflow strategy is proposed to reduce off-chip data transfer of feature maps. Also, multi-level buffers are proposed to keep the most needed data on-chip (in block RAM) and avoid off-chip data reorganization and reloading. Finally, a computation core is proposed to operate the depthwise, pointwise, and max-pooling computation as soon as the data arrives without reorganization, which suits the real-life scenario of the data arriving in sequence. In our experiment, FitNN is implemented on two FPGA-based platforms (both at150 MHz), Ultra96-V2 and PYNQ-Z1, for drone-based object detection with batch size=1. The results show that FitNN achieves 15 frames per second (FPS) on Ultra96-V2, with power consumption of 4.69 W. On PYNQ-Z1, FitNN achieves 9 FPS with 1.9 W of power consumption. Compared with the previous FPGA-based implementation of iSmart2 CNN, FitNN increases the efficiency (FPS/W) by 2.37 times.Keywords
This publication has 44 references indexed in Scilit:
- A CNN Accelerator on FPGA Using Depthwise Separable ConvolutionIEEE Transactions on Circuits and Systems II: Express Briefs, 2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Scaling for edge inference of deep neural networksNature Electronics, 2018
- Techology trend of edge AI2018
- A 65nm 1Mb nonvolatile computing-in-memory ReRAM macro with sub-16ns multiply-and-accumulate for binary DNN AI edge processors2018
- FP-BNN: Binarized neural network on FPGANeurocomputing, 2018
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- Small neural nets are beautiful2017
- FP-DNN: An Automated Framework for Mapping Deep Neural Networks onto FPGAs with RTL-HLS Hybrid Templates2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016