Hummingbird: A Smaller and Faster Large Language Model Accelerator on Embedded FPGA
Authors:
- 26 October 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Deploying large language models (LLMs) on embedded devices remains a significant research challenge due to the high computational and memory demands of LLMs and the limited hardware resources available in such environments. While embedded FPGAs have demonstrated performance and energy efficiency in traditional deep neural networks, their potential for LLM inference remains largely unexplored. Recent efforts to deploy LLMs on FPGAs have primarily relied on large, expensive cloud-grade hardware and have only shown promising results on relatively small LLMs, limiting their real-world applicability. In this work, we present Hummingbird, a novel FPGA accelerator designed specifically for LLM inference on embedded FPGAs. Hummingbird is smaller—targeting embedded FPGAs such as the KV260 and ZCU104 with 67% LUT, 39% DSP, and 42% power savings over existing research. Hummingbird is stronger—targeting LLaMA3-8B and supporting longer contexts, overcoming the typical 4GB memory constraint of embedded FPGAs through offloading strategies. Finally, Hummingbird is faster—achieving 4.8 tokens/s and 8.6 tokens/s for LLaMA3-8B on the KV260 and ZCU104 respectively, with 93-94% model bandwidth utilization, outperforming the prior 4.9 token/s for LLaMA2-7B with 84% bandwidth utilization baseline. We further demonstrate the viability of industrial applications by deploying Hummingbird on a cost-optimized Spartan UltraScale FPGA, paving the way for affordable LLM solutions at the edge.Keywords
All Related Versions
Funding Information
- Chinese Academy of Sciences
This publication has 25 references indexed in Scilit:
- Efficient Hardware Architecture Design for Rotary Position Embedding of Large Language ModelsIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2025
- Revealing Untapped DSP Optimization Potentials for FPGA-Based Systolic Matrix Engines2024
- An FPGA-Based Efficient Streaming Vector Processing Engine for Transformer-Based Models2024
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation2022
- Unexpected Diversity: Quantitative Memory Analysis for Zynq UltraScale+ Systems2019
- Scaling the Cascades: Interconnect-Aware FPGA Implementation of Machine Learning Problems2019
- Sigmoid-weighted linear units for neural network function approximation in reinforcement learningNeural Networks, 2018
- Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGAIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
- A 1.2 V 30 nm 3.2 Gb/s/pin 4 Gb DDR4 SDRAM With Dual-Error Detection and PVT-Tolerant Data-Fetch SchemeIEEE Journal of Solid-State Circuits, 2012