Ben Feinberg

In modern generative-AI workloads, matrix–vector/matrix–matrix multiplications (MatMul) dominate the compute and energy cost. Achieving dramatic reductions in energy per token therefore requires a novel, specialized hardware that is co-designed across materials, devices, interconnects, circuits, and architectures rather than optimized at any single layer in isolation. In this Perspectives article, we argue that predictive (first-principles, fitting-parameter-free) device and interconnect simulations can close the loop between nanoscale physics and workload-level metrics, enabling the identification of device/interconnect operating regimes that plausibly support orders-of-magnitude improvements in energy efficiency of AI accelerators.

Resistive random-access memory (RRAM)-based in-memory computing (R-IMC) architectures achieve energy efficiency by avoiding data movement for matrix-vector computation. However, R-IMC suffers from reliability challenges, such as programming errors, conductance drift, and stuck-at faults, which introduce noise and degrade performance. Several approaches have been proposed in the literature to enhance reliability; however, these solutions are limited to a few specific types of non-idealities, and they cannot be applied to large models, pre-trained models, or R-IMC architectures without introducing considerable computation and memory overhead. We propose a design-time solution that uses a mixture of fault-tolerant experts by adapting pre-trained models via low-rank adaptation. We train non-idealities-specific experts, which are merged with the base weights before mapping, ensuring no added cost yet reliable R-IMC inferencing. For a range of vision and language tasks with transformer models, we show that our approach improves inferencing accuracy by up to 76% under a mixture of non-idealities, enabling reliable mapping and inferencing of vision–language models with no additional cost at runtime.