Joon-Sung Yang

Deep Neural Networks (DNNs) have achieved out standing performance in various fields. However, as DNNs continue to grow in size, their performance is hindered by the memory wall problem, where data memory bandwidth becomes a bottleneck. Resistive Random Access Memory (ReRAM)-based DNN accelerators offer promising solutions to this challenge, enhancing data processing efficiency. However, these architectures face reliability issues due to cell immaturity, notably device conductance variation and offset currents in High Resistance State (HRS) cells, which compromise DNN accuracy and processing performance. This paper addresses the reliability issues of the ReRAM-based computing-in-memory (CIM) accelerator. In particular, this paper introduces a novel design methodology of shared-exponent deployment on the accelerator, enhancing variation tolerance through fine-grained quantization. In addition, this paper proposes a variation-resilient data encoding to further improve the robustness in combination with the proposed design and introduces a programming scheme for the offset current compensation. Experimental results conducted on various DNN models and datasets reveal that the proposed techniques outperform existing methods in variation tolerance and significantly improves performance and energy efficiency compared to the conventional baseline, even under high variation environments.

In electronic design automation (EDA), simulation models are often non-differentiable, and many design choices are discrete. As a result, greedy optimization methods based on numerical gradients are widely used, although they often lead to suboptimal solutions. In contrast, analytical methods may provide better solutions but require significant research effort. Reinforcement learning (RL) has been employed to address this problem; however, RL also suffers from notorious sample inefficiency, which is exaggerated in EDA because data sampling in EDA is very expensive due to slow simulations. This article proposes an alternative to RL for EDA, namely analytic gradient descent (AGD). Our method starts with a differentiable performance model, which can be either a learned surrogate or a static model. It then applies transformations similar to Shannon decomposition for each design variable in the performance model. Finally, one design option for each variable is selected using a one-hot variable, which is trained via a straight-through estimator (STE) through gradient descent. We demonstrate AGD on the well-known gate sizing problem using both a learned surrogate and a static model across 20 industrial benchmark circuits. Our experimental results show that the proposed method can outperform a several-decade-old commercial tool in the gate sizing task for 19 out of the 20 circuits.

Large language models (LLMs), which have emerged from advances in natural language processing (NLP), enable chatbots, virtual assistants, and numerous domain-specific applications. These models, often comprising billions of parameters, leverage the Transformer architecture and Attention mechanisms to process context effectively and address long-term dependencies more efficiently than earlier approaches, such as recurrent neural networks (RNNs). Notably, since the introduction of Llama, the architectural development of LLMs has significantly converged, predominantly settling on a Transformer-based decoder-only architecture. The evolution of LLMs has been driven by advances in high-bandwidth memory, specialized accelerators, and optimized architectures, enabling models to scale to billions of parameters. However, it also introduces new challenges: meeting compute and memory efficiency requirements across diverse deployment targets, ranging from data center servers to resource-constrained edge devices. To address these challenges, we survey the evolution of LLMs at two complementary levels: architectural trends and their underlying operational mechanisms. Furthermore, we quantify how hyperparameter settings influence inference latency by profiling kernel-level execution on a modern GPU architecture. Our findings reveal that identical models can exhibit varying performance based on hyperparameter configurations and deployment contexts, emphasizing the need for scalable and efficient solutions. The insights distilled from this analysis guide the optimization of performance and efficiency within these converged LLM architectures, thereby extending their applicability across a broader range of environments.

With the increasing demand for on-device Artificial Intelligence (AI), various compression schemes have been proposed for DNN models to be efficiently executed on mobile devices. Although various studies have proposed compression schemes, their compatibility with mobile device memory (i.e., LPDDR) is not considered; LPDDR is prone to error as it operates at low voltage due to its strict power constraint. Currently, DRAM vendors adopt an ECC engine with SEC(136,128) code inside each LPDDR bank (i.e., IECC) for reliable operation. While IECC enhances reliability, it lowers performance due to Read-Modify-Write (RMW) and parity storage costs. Thus, for both power-efficient and reliable DNN operation in mobile environments, this paper introduces LOCo, a DNN weight compression scheme with 3-staged protection. LOCo reduces the IECC engine operation granularity from SEC(136,128) to SECDED(72,64), thus eliminating power-intensive internal reads but enhancing reliability. To mitigate the storage overhead, this paper proposes a compression scheme with protection based on the characteristics of DNN weights. Our evaluations show that LOCo provides a power reduction of 16.94%, latency reduction of 16.81%, and energy reduction of 30.81% when compared to conventional LPDDR with SEC engine, while demonstrating robustness under 17959X more bit error rate when compared to existing compression scheme.

In-memory Computing (IMC) using Phase Change Memory (PCM) has proven to be effective for efficient processing of Deep Neural Networks (DNNs). However, with the use of multi-level cell PCM (MLC-PCM) in NVMs-based accelerators, errors due to resistance drift in MLC-PCM can severely degrade the DNNs accuracy. In this paper, an analysis of the impact of resistance drift errors on accuracy of MLC-PCM based DNN accelerator shows that the drift errors alone can significantly impact the accuracy. This paper proposes Hydra, which is a hybrid resistance drift resilient architecture for MLC-PCM based DNN accelerators which use IMC for efficient computations. Hydra utilizes Tri-level cell PCM, which has a negligible resistance drift error rate, to store the critical bits of DNNs parameters and MLC-PCM (4-level cell), which has a higher error rate (but offers more storage density), for the non-critical bits. Experimental results on various DNN architectures, configurations and datasets show that, with the presence of resistance drift errors in PCM, Hydra can maintain the baseline accuracy of DNNs for up to 1 year (resistance drift is time-dependent), whereas conventional drift tolerance techniques lead to a significant accuracy drop in just a few seconds.