Abstract
Quantization enables efficient large language model (LLM) inference by reducing memory and computation cost, but most existing methods sacrifice hardware simplicity through floating-point fallback, dequantization logic, or metadata routing. These overheads disrupt datapath regularity and limit gains in timing and area, especially in multiply-accumulate (MAC)-heavy transformer layers. We propose LogiMAC, a hardware-aligned quantization scheme that encodes weights in the log domain using shift-based arithmetic. LogiMAC embeds all decoding logic within the MAC unit, eliminates runtime scaling, and supports sublevel refinement without multipliers or look-up tables. The resulting datapath supports both shift directions with optional gating for unused paths. We evaluate LogiMAC across seven modern LLMs (ranging from 2B to 8B parameters) and compare it to strong baselines including SmoothQuant, GOBO, and SpinQuant. Our results show that while functional accuracy remains competitive, LogiMAC reduces area by $1.8\times $ and dynamic power by $5.9\times $ (in power-gated mode) compared to standard integer baselines. Timing analysis reveals over 50% of LogiMAC paths complete within just 12% of the normalized delay seen in other schemes. Validated over 34 million generated tokens, our findings demonstrate that accurate low-bit quantization can coexist with extreme hardware efficiency.
Funding Information
  • National Science Foundation (CNS-2106237)