An Agile Systolic Array-Based Hardware Accelerator for Scalable Multi-Head Self-Attention
- 5 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 18 (2) , 90-93
Abstract
As attention mechanisms find increasing application in computer vision, numerous neural networks are now integrating convolutional neural networks (CNNs) with attention layers to enhance performance. As a typical representative, the Bottleneck Transformer (BoT) network effectively balances the extraction of local features and modeling of global context, leading to significant performance improvements. However, the multi-head self-attention (MHSA) layer of BoT deviates from conventional MHSA by incorporating a wider range of matrix operations, posing challenges in computational complexity and memory bandwidth. This letter introduces a customized accelerator for the MHSA layer of BoT. The accelerator employs a configurable systolic array architecture, designed to support various matrix operations, including matrix transposition, addition, multiplication, and a hardware-optimized Softmax, facilitating hybrid scalable MHSA layers. Implemented on an FPGA, the proposed accelerator achieves a performance of 143.98 GOPS at a frequency of 320 MHz. When compared to CPU implementations, our design offers a 7.18× reduction in inference time.Keywords
Funding Information
- National Natural Science Foundation of China (62271300)
- National Natural Science Foundation of China (12141107)
- Shanghai Minhang Science and Technology Commission (24DP1501100)
- Shanghai Minhang Science and Technology Commission (24DP1500600)
This publication has 12 references indexed in Scilit:
- HALO: Communication-Aware Heterogeneous 2.5-D System for Energy-Efficient LLM Execution at EdgeIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2024
- Calabash: Accelerating Attention Using a Systolic Array Chain on FPGAs2023
- Unified Accelerator for Attention and Convolution in Inference Based on FPGA2023
- LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference2021
- Bottleneck Transformers for Visual Recognition2021
- SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning2021
- End-to-End Object Detection with Transformers2020
- Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators2020
- A High-Speed and Low-Complexity Architecture for Softmax Function in Deep Learning2018
- Deep Residual Learning for Image Recognition2016