HotBa: A Heterogeneous Mamba Accelerator with Δ-Guided Early Rejection for Speculative Decoding
Open Access
- 5 August 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Abstract
Mamba has emerged as a promising alternative to Transformers for on-device LLM inference, offering linear complexity and eliminating KV-cache. However, autoregressive decoding reloads full model weights every token, accounting for 93.2% of total inference energy, with no existing accelerator resolving this overhead. We present HotBa, a heterogeneous Mamba accelerator that reduces per-token weight transfer by 82% and redundant computation by 59% via Δ-guided early rejection for wide-tree speculative decoding in Mamba, while a heterogeneous INT8/FP16 core and tree management unit achieve 40.4× area efficiency and 5.18× SSM speedup with 0.4% area overhead. Synthesized in 28nm CMOS, HotBa achieves 75.32 tokens/s with 1.65× speedup and 7.55× energy efficiency over the state-of-the-art Mamba accelerator.Funding Information
- Institute of Information & Communications Technology Planning & Evaluation (IITP) / Korea government (MSIT) (RS-2025-09942968)
- Institute of Information & Communications Technology Planning & Evaluation (IITP) / Korea government (MSIT) (RS-2024-00399817)
- IC Design Education Center (IDEC), Korea (EDA tool)
This publication has 7 references indexed in Scilit:
- MARCA-v2: Mamba Accelerator With Complementary State-Space Model Sparsity and Reconfigurable ArchitectureIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2025
- A Quantitative Analysis of Mamba-2-Based Large Language Model: Study of State Space DualityIEEE Computer Architecture Letters, 2025
- A 2.67 mJ/frame Video Mamba Accelerator with Importance-aware Redundancy Elimination and SSM Computing Reformulation2025
- MARCA: Mamba Accelerator with Reconfigurable Architecture2024
- WinoGrandeCommunications of the ACM, 2021
- PIQA: Reasoning about Physical Commonsense in Natural LanguageProceedings of the AAAI Conference on Artificial Intelligence, 2020
- The LAMBADA dataset: Word prediction requiring a broad discourse context2016