BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference
Preprint
Abstract
Compute-in-Read-Only-Memory (CiROM) accelerators offer outstanding energy efficiency for CNNs by eliminating runtime weight updates. However, their scalability to Large Language Models (LLMs) is fundamentally constrained by their vast parameter sizes. Notably, LLaMA-7B - the smallest model in LLaMA series - demands more than 1,000 cm2 of silicon area even in advanced CMOS nodes. This paper presents BitROM, the first CiROM-based accelerator that overcomes this limitation through co-design with BitNet's 1.58-bit quantization model, enabling practical and efficient LLM inference at the edge. BitROM introduces three key innovations: 1) a novel Bidirectional ROM Array that stores two ternary weights per transistor; 2) a Tri-Mode Local Accumulator optimized for ternary-weight computations; and 3) an integrated Decode-Refresh (DR) eDRAM that supports on-die KV-cache management, significantly reducing external memory access during decoding. In addition, BitROM integrates LoRA-based adapters to enable efficient transfer learning across various downstream tasks. Evaluated in 65nm CMOS, BitROM achieves 20.8 TOPS/W and a bit density of 4,967 kB/mm2 - offering a 10x improvement in area efficiency over prior digital CiROM designs. Moreover, the DR eDRAM contributes to a 43.6% reduction in external DRAM access, further enhancing deployment efficiency for LLMs in edge applications.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- ASiM: Modeling and Analyzing Inference Accuracy of SRAM-Based Analog CiM CircuitsIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2025
- Hybrid SRAM/ROM Compute-in-Memory Architecture for High Task-Level Energy Efficiency in Transformer Models With 8928-kb/mm² Density in 28nm CMOSIEEE Journal of Solid-State Circuits, 2025
- Cramming More Weight Data Onto Compute-in-Memory Macros for High Task-Level Energy Efficiency Using Custom ROM With 3984-kb/mm2 Density in 65-nm CMOSIEEE Journal of Solid-State Circuits, 2023
- GC-eDRAM With Body-Bias Compensated Readout and Error Detection in 28-nm FD-SOIIEEE Transactions on Circuits and Systems II: Express Briefs, 2019
- DRAM Refresh Mechanisms, Penalties, and Trade-OffsIEEE Transactions on Computers, 2015