Calabash: Accelerating Attention Using a Systolic Array Chain on FPGAs
Authors:
- 4 September 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 242-247
Abstract
In recent years, attention mechanism has achieved remarkable performance in natural language processing and computer vision applications, at the expense of high computation cost. FPGAs have been demonstrated to be an effective hardware platform for various AI applications. However, the attention mechanism involves complex data dependency, which makes FPGA acceleration difficult. In this paper, we propose Calabash, an FPGA accelerator for attention-based applications. We design a chain of two systolic arrays, applying the same dataflow. Then, we design two scheduling techniques for different matrices to ensure the intermediate matrix can be cached in the on-chip memory. Finally, we develop analytical models for resource utilization estimation, workload balancing, and latency prediction to guide design space exploration. Experiments show that Calabash achieves 1.76 TOP/s, 1.06 TOP/s on Xilinx VU9P and ZCU102 platforms, yielding an average 50.1X and 3.94X energy-efficiency improvement compared with CPU and GPU, respectively.Keywords
Funding Information
- National Natural Science Foundation of China (T2293700,T2293701)
This publication has 23 references indexed in Scilit:
- LUTNet: Rethinking Inference in FPGA Soft Logic2019
- Speedy2019
- Evaluating Fast Algorithms for Convolutional Neural Networks on FPGAsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019
- PolySA2018
- A High-Speed and Low-Complexity Architecture for Softmax Function in Deep Learning2018
- SpWA: An Efficient Sparse Winograd Convolutional Neural Networks Accelerator on FPGAs2018
- A Read-Write Memory Network for Movie Story Understanding2017
- Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs2017
- A reconfigurable fabric for accelerating large-scale datacenter services2014
- Chisel2012