XVDPU: A High-Performance CNN Accelerator on the Versal Platform Powered by the AI Engine
Open Access
- 13 March 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Reconfigurable Technology and Systems
- Vol. 17 (2) , 1-24
Abstract
Today, convolutional neural networks (CNNs) are widely used in computer vision applications. However, the trends of higher accuracy and higher resolution generate larger networks. The requirements of computation or I/O are the key bottlenecks. In this article, we propose XVDPU: the AI Engine (AIE)-based CNN accelerator on Versal chips to meet heavy computation requirements. To resolve the IO bottleneck, we adopt several techniques to improve data reuse and reduce I/O requirements. An arithmetic logic unit is further proposed that can better balance resource utilization, new feature support, and efficiency of the whole system. We have successfully deployed more than 100 CNN models with our accelerator. Our experimental results show that the 96-AIE-core implementation can achieve 1,653 frames per second (FPS) for ResNet50 on VCK190, which is 9.8× faster than the design on ZCU102 running at 168.5 FPS. The 256-AIE-core implementation can further achieve 4,050 FPS. We propose a tilling strategy to achieve feature-map-stationary for high-definition CNN with the accelerator, achieving 3.8× FPS improvement on the residual channel attention network and 3.1× on super-efficient super-resolution. This accelerator can also solve the 3D convolution task in disparity estimation, achieving end-to-end performance of 10.1 FPS with all the optimizations.Keywords
This publication has 19 references indexed in Scilit:
- Deep Learning for Single Image Super-Resolution: A Brief ReviewIEEE Transactions on Multimedia, 2019
- Deep learning methods in real-time image super-resolution: a surveyJournal of Real-Time Image Processing, 2019
- Xilinx Adaptive Compute Acceleration Platform2019
- Sigmoid-weighted linear units for neural network function approximation in reinforcement learningNeural Networks, 2018
- Image Super-Resolution Using Very Deep Residual Channel Attention Networks2018
- RoPose: CNN-based 2D Pose Estimation of Industrial Robots2018
- Pyramid Stereo Matching Network2018
- Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle EnvironmentIEEE Transactions on Industrial Informatics, 2018
- Towards a Uniform Template-based Architecture for Accelerating 2D and 3D CNNs on FPGA2018
- A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation2016