Memristor-Based Circuit Implementation and Circuitry Optimized Algorithm for Mamba Language Network
- 22 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems I: Regular Papers
- Vol. 73 (1) , 425-438
Abstract
Language networks are crucial in artificial intelligence, with the novel Mamba architecture significantly reducing computations and consumption compared to the traditional transformer network. However, a full-circuit implementation of the Mamba network has not been proposed due to the complexity of computations and data storage. Additionally, optimized hardware-aware parallel algorithms for Mamba inference in circuits remain undeveloped. This work addresses these challenges by presenting a memristor-based full-circuit implementation of the Mamba network and introducing a computing-in-memory parallel-aware algorithm tailored for circuit-level inference. The implementation includes: 1) Standard 1T1M memristor crossbar and depthwise separable convolution memristor crossbar for different convolutions. 2) Computing-in-memory implicit latent state circuits for the computation and transition of latent states. 3) Functional circuits for SiLU activation, RMS normalization, and multi-layer multiply-accumulate operations. 4) Optimized algorithm and circuit implementation for hardware-aware inference, achieving parallel scanning and hardware awareness in circuits. The proposed circuit enables analog signal computations and eliminates redundant analog-to-digital conversions and intermediate storage. A basic single-sentence generation task was simulated in PSPICE, validating the circuit’s correctness. Analyses of analog computation accuracy, circuit stability, and power consumption demonstrate the proposed circuit’s advantages, highlighting its potential as a fundamental module for large-scale circuit integration and complex text generation tasks.Keywords
Funding Information
- National Key Research and Development Program of China (2023YFB4402402)
- National Natural Science Foundation of China (62474074)
This publication has 31 references indexed in Scilit:
- A crossbar array of magnetoresistive memory devices for in-memory computingNature, 2022
- The Future of Memristors: Materials Engineering and Neural NetworksAdvanced Functional Materials, 2020
- Brain-inspired computing with memristors: Challenges in devices, circuits, and systemsApplied Physics Reviews, 2020
- Efficient and self-adaptive in-situ learning in multilayer memristor neural networksNature Communications, 2018
- Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuitsNature Communications, 2018
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- Emulation of synaptic metaplasticity in memristorsNanoscale, 2016
- Deep learningNature, 2015
- A survey on the application of recurrent neural networks to statistical language modelingComputer Speech & Language, 2015
- The missing memristor foundNature, 2008