TRAC: Compilation-Based Design of Transformer Accelerators for FPGAs
Authors:
- 1 August 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 17-23
Abstract
Transformer-type Neural Networks (NNs) have shown impressive accuracy numbers in Natural Language Processing (NLP) applications where Recurrent Neural Networks (RNNs) have been in use before, even surpassing them. However, differing considerably from common types of NNs, existing accelerator designs, particularly for Field-Programmable Gate Arrays (FPGAs), cannot be used to implement them. Previous research has shown FPGAs to be platforms superior to CPUs and even GPUs for accelerating NNs when it comes to energy efficiency. Following the development of automated compiler-based design flows for NNs, there is still a lack of such an approach for transformers and FPGA targets. In this realm, this paper presents a novel compiler called TRAC as well as a library of operators and modules for implementing transformer accelerators on FPGAs. Based on optimization and code generation settings in the compiler using an integrated approach combining weight compression techniques with according adaptations of the accelerator modules, a design space of accelerators is defined and explored. For each design, a system-level data path and control unit architecture is generated, which integrates module-level designs using hierarchical High-Level Synthesis (HLS). We evaluate our implementation for the BERT network and provide results regarding the trade-off between execution time, accuracy, and FPGA resource usage.Keywords
This publication has 17 references indexed in Scilit:
- Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning2021
- NPE: An FPGA-based Overlay Processor for Natural Language Processing2021
- FTRANS2020
- Flexible Communication Avoiding Matrix Multiplication on FPGA with High-Level Synthesis2020
- Efficient Mapping of CNNs onto Tightly Coupled Processor ArraysJournal of Computers, 2019
- FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A ReviewIEEE Access, 2019
- FINN- RACM Transactions on Reconfigurable Technology and Systems, 2018
- Fast inference of deep neural networks in FPGAs for particle physicsJournal of Instrumentation, 2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding2018