Variation-Aware Memristor-Based Analog Accelerator for Vision Transformer
Open Access
- 8 March 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (5) , 1116
Abstract
Vision transformers (ViTs) have emerged as one of the most popular computer vision models, achieving remarkable performance in image recognition. However, ViTs require large-scale, high-dimensional matrix computations, and traditional digital accelerators, such as graphics processing units (GPUs), have memory bandwidth limitations, leading to higher latency, increased energy consumption, and larger area. To address this challenge, this paper proposes a memristor-based analog accelerator that leverages memristor crossbar arrays for in-memory computing, reducing data movement and improving computational efficiency. Considering the non-ideal characteristics of memristor devices and the influence of analog circuitry, we incorporate Gaussian-distributed analog computation error at each step and memristor non-ideality modeling into the ViT inference to enable realistic evaluation under hardware-level conditions. Experimental evaluation on ImageNet-1k dataset with TIMM-pretrained ViT models shows that the proposed analog accelerator can achieve the same Top-1 accuracy as a custom-designed 5 nm digital baseline accelerator, even with ~35% analog computation error and ~10% memristor conductance variation injected at each step. Compared to the digital counterpart, the proposed design achieves an 11.9× reduction in energy-delay product (EDP) and a 137.2× reduction in energy-delay-area product (EDAP).Keywords
Funding Information
- U.S. Department of Energy (DE-SC0022881)
- National Science Foundation (CCF-2219753)
This publication has 26 references indexed in Scilit:
- An adaptive fused domain-cycling variational generative adversarial network for machine fault diagnosis under data scarcityInformation Fusion, 2025
- A dynamic collaborative adversarial domain adaptation network for unsupervised rotating machinery fault diagnosisReliability Engineering & System Safety, 2024
- Conditional distribution-guided adversarial transfer learning network with multi-source domains for rolling bearing fault diagnosisAdvanced Engineering Informatics, 2023
- Thousands of conductance levels in memristors integrated on CMOSNature, 2023
- Multimodal Neuromorphic Sensory-Processing System With Memristor Circuits for Smart Home ApplicationsIEEE Transactions on Industry Applications, 2022
- Full-Circuit Implementation of Transformer Network Based on MemristorIEEE Transactions on Circuits and Systems I: Regular Papers, 2022
- Memristor-Based Analog Recursive Computation Circuit for Linear Programming OptimizationIEEE Journal on Exploratory Solid-State Computational Devices and Circuits, 2020
- Memory devices and applications for in-memory computingNature Nanotechnology, 2020
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Benchmarking of Beyond-CMOS Exploratory Devices for Logic Integrated CircuitsIEEE Journal on Exploratory Solid-State Computational Devices and Circuits, 2015