Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application
Open Access
- 29 June 2022
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 14 (13) , 3130
Abstract
In recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a challenging problem, especially for the spaceborne scenario where resources and power consumption are limited. To tackle this problem, this paper proposes an automatic CNN deployment solution on resource-limited field-programmable gate arrays (FPGAs) for spaceborne remote sensing applications. Firstly, a series of hardware-oriented optimization methods are proposed to reduce the complexity of the CNNs. Secondly, a hardware accelerator is designed. In this accelerator, a reconfigurable processing engine array with efficient convolutional computation architecture is used to accelerate CNN-based algorithms. Thirdly, to bridge the optimized CNNs and hardware accelerator, a compilation toolchain is introduced into the deployment solution. Through the automatic conversion from CNN models to hardware instructions, various networks can be deployed on hardware in real-time. Finally, we deployed an improved VGG16 network and an improved YOLOv2 network on Xilinx AC701 to evaluate the effectiveness of the proposed deployment solution. The experiments show that with only 3.407 W power consumption and 94 DSP consumption, our solution achieves 23.06 giga operations per second (GOPS) throughput in the improved VGG16 and 22.17 GOPS throughput in the improved YOLOv2. Compared to the related works, the DSP efficiency of our solution is improved by 1.3–2.7×.Keywords
Funding Information
- National Science Fund for Distinguished Young Scholars (2018-JCJQ-ZQ-046, B0201)
- Civil Aviation Program (2018-JCJQ-ZQ-046, B0201)
This publication has 67 references indexed in Scilit:
- HPT: A High Spatial Resolution Multispectral Sensor for Microsatellite Remote SensingSensors, 2018
- Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGAIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- YOLO9000: Better, Faster, Stronger2017
- Remote Sensing Image Scene Classification: Benchmark and State of the ArtProceedings of the IEEE, 2017
- SSD: Single Shot MultiBox Detector2016
- Cambricon: An Instruction Set Architecture for Neural Networks2016
- A Novel GPU-Based Efficient Approach for Convolutional Neural Networks with Small FiltersJournal of Signal Processing Systems, 2016
- Comparison of Laser and Stereo Optical, SAR and InSAR Point Clouds from Air- and Space-Borne Sources in the Retrieval of Forest Inventory AttributesRemote Sensing, 2015
- SP-CNN: A Scalable and Programmable CNN-Based AcceleratorIEEE Micro, 2015
- Linear scan register allocationACM Transactions on Programming Languages and Systems, 1999