Sangjin Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sangjin Kim has a total of 46 co-authors. Here are the top co-authors by number of shared publications:
- Hoi-Jun Yoo
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
34 shared publications - Sangyeob Kim
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
21 shared publications - Donghyeon Han
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA
13 shared publications - Soyeon Um
Massachusetts Institute of Technology (MIT)
12 shared publications - Wooyoung Jo
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
10 shared publications - Soyeon Kim
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
9 shared publications
Latest Publications
- Open Access
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
- 26 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessSliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
- 22 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. PP (00189200) , 1-13
MIDAS: An Energy-Efficient Microscaling Quantization-Based Digital CIM Accelerator With Spatio-Temporal Cyclic Alignment for Generative AI
Digital compute-in-memory (DCIM) and the microscaling (MX) data format have emerged as promising hardware and algorithmic solutions, respectively, for energy-efficient generative AI (GenAI) processing. However, their combination introduces challenges, including redundant computation from mantissa alignment, excessive exponent handling overhead, and accuracy loss from static approximation schemes that fail to accommodate block-wise dynamic bit-significance. In this article, we present MIDAS, an MX-optimized DCIM-based GenAI accelerator with three key features: 1) spatio-temporal cyclic alignment (STCA) with sign-magnitude-based assembled column addition (SACA) enables compact weight mapping, high input reusability, and low-latency computation, achieving $1.72\times $ higher macro energy efficiency; 2) MX-aware minimum exponent post-detection (MEPD) eliminates redundant maximum exponent detection by exploiting the inherent characteristics of MX quantization, reducing exponent handling power, area, and latency by 68.8%, 63.2%, and 22.5%, respectively; and 3) sense amplifier-based dynamic bit (SADB)-significance handling with shared scale-aware channel regrouping (SSCR) dynamically adjusts approximation bit-positions according to each block’s shared scale, improving energy efficiency by 27.3% while minimizing accuracy loss. MIDAS is fabricated in Samsung 28-nm CMOS technology and occupies a 7.56 mm2 die area. MIDAS achieves macro energy and area efficiencies of 96.3 TFLOPS/W and 0.69 TFLOPS/mm2, respectively, under MXFP4. Compared to state-of-the-art DCIMs, MIDAS demonstrates $1.58\times $ higher system energy efficiency and $3.32\times $ higher macro energy efficiency, validating its effectiveness as an MX-optimized AI accelerator for next-generation GenAI applications.
- 22 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. PP (00189200) , 1-13
MIDAS: An Energy-Efficient Microscaling Quantization-Based Digital CIM Accelerator With Spatio-Temporal Cyclic Alignment for Generative AI
Digital compute-in-memory (DCIM) and the microscaling (MX) data format have emerged as promising hardware and algorithmic solutions, respectively, for energy-efficient generative AI (GenAI) processing. However, their combination introduces challenges, including redundant computation from mantissa alignment, excessive exponent handling overhead, and accuracy loss from static approximation schemes that fail to accommodate block-wise dynamic bit-significance. In this article, we present MIDAS, an MX-optimized DCIM-based GenAI accelerator with three key features: 1) spatio-temporal cyclic alignment (STCA) with sign-magnitude-based assembled column addition (SACA) enables compact weight mapping, high input reusability, and low-latency computation, achieving $1.72\times $ higher macro energy efficiency; 2) MX-aware minimum exponent post-detection (MEPD) eliminates redundant maximum exponent detection by exploiting the inherent characteristics of MX quantization, reducing exponent handling power, area, and latency by 68.8%, 63.2%, and 22.5%, respectively; and 3) sense amplifier-based dynamic bit (SADB)-significance handling with shared scale-aware channel regrouping (SSCR) dynamically adjusts approximation bit-positions according to each block’s shared scale, improving energy efficiency by 27.3% while minimizing accuracy loss. MIDAS is fabricated in Samsung 28-nm CMOS technology and occupies a 7.56 mm2 die area. MIDAS achieves macro energy and area efficiencies of 96.3 TFLOPS/W and 0.69 TFLOPS/mm2, respectively, under MXFP4. Compared to state-of-the-art DCIMs, MIDAS demonstrates $1.58\times $ higher system energy efficiency and $3.32\times $ higher macro energy efficiency, validating its effectiveness as an MX-optimized AI accelerator for next-generation GenAI applications.
- 26 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 61 (1) , 168-182
EdgeDiff: Energy-Efficient Multi-Modal Few-Step Diffusion Model Accelerator Using Mixed-Precision and Reordered Group Quantization
Recent advances in diffusion models (DMs)—such as few-step denoising and multi-modal conditioning—have significantly improved computational efficiency and functional flexibility, but they also introduce new hardware challenges. In particular, the elimination of inter-timestep redundancy, increased encoder/decoder workload, and heightened sensitivity to quantization demand a new class of accelerator. We present EdgeDiff, the first processor to support end-to-end, few-step, and multi-modal DM inference. EdgeDiff introduces a unified solution named condition-aware reordered group mixed precision (CRMP) with several novel microarchitectures: compress-and-add (CAA) processing elements (PEs) with bit-shuffle trees (BSTs) for efficient low-bit multiply-accumulate (MAC), a tiered accumulation unit (TAU) to reduce floating-point (FP) accumulation energy, and a grid-based quantization unit (GQU) to eliminate expensive FP division. Fabricated in 28-nm CMOS, EdgeDiff achieves up to 34.4-TOPS/W energy efficiency and reduces generation energy to 418.4 mJ/image for one-step text-to-image (T2I) generation— $3.3\times $ lower than prior state of the art. Despite aggressive quantization, EdgeDiff maintains output quality comparable to FP inference across Fréchet Inception Distance (FID), contrastive language–image pretraining (CLIP), and peak signal-to-noise ratio (PSNR) metrics, establishing it as a compelling solution for energy-efficient, real-time generative artificial intelligence (AI) on edge platforms.
- 26 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 61 (1) , 168-182
EdgeDiff: Energy-Efficient Multi-Modal Few-Step Diffusion Model Accelerator Using Mixed-Precision and Reordered Group Quantization
Recent advances in diffusion models (DMs)—such as few-step denoising and multi-modal conditioning—have significantly improved computational efficiency and functional flexibility, but they also introduce new hardware challenges. In particular, the elimination of inter-timestep redundancy, increased encoder/decoder workload, and heightened sensitivity to quantization demand a new class of accelerator. We present EdgeDiff, the first processor to support end-to-end, few-step, and multi-modal DM inference. EdgeDiff introduces a unified solution named condition-aware reordered group mixed precision (CRMP) with several novel microarchitectures: compress-and-add (CAA) processing elements (PEs) with bit-shuffle trees (BSTs) for efficient low-bit multiply-accumulate (MAC), a tiered accumulation unit (TAU) to reduce floating-point (FP) accumulation energy, and a grid-based quantization unit (GQU) to eliminate expensive FP division. Fabricated in 28-nm CMOS, EdgeDiff achieves up to 34.4-TOPS/W energy efficiency and reduces generation energy to 418.4 mJ/image for one-step text-to-image (T2I) generation— $3.3\times $ lower than prior state of the art. Despite aggressive quantization, EdgeDiff maintains output quality comparable to FP inference across Fréchet Inception Distance (FID), contrastive language–image pretraining (CLIP), and peak signal-to-noise ratio (PSNR) metrics, establishing it as a compelling solution for energy-efficient, real-time generative artificial intelligence (AI) on edge platforms.
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 34 (1) , 118-129
An Energy-Efficient High Resolution Vision Transformer Processor Exploiting Token Similarity Beyond Token Merging
Token merging is a widely adopted technique in vision transformers (ViTs) to reduce computation with minimal accuracy degradation. However, in tasks that require fine-grained outputs, such as text-unified object recognition, aggressive merging reaches a limit due to accuracy loss. To address this, we introduce a method that exploits residual token similarity beyond the merging threshold using difference computing, which significantly improves energy efficiency. The proposed processor integrates the post merging residual difference (PMRD) algorithm, which first reduces 34% of tokens through merging, then applies difference computing to the remaining tokens. This results in a 31% reduction in effective bit operations by leveraging residual similarity. To further exploit the resulting bit-level sparsity, the processor employs sparsity-separated accumulation with sign-magnitude representation (SSA-SM) and a bi-directional dynamic memory allocation (BDA) to optimize both computation and memory access. These enhancements yield an additional 29.4% computational energy savings. Overall, the system achieves a $2.22\times $ improvement in energy efficiency and delivers 7.23 TOPS/W on the MS COCO dataset for text-unified object recognition.
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
- Vol. 34 (1) , 118-129
An Energy-Efficient High Resolution Vision Transformer Processor Exploiting Token Similarity Beyond Token Merging
Token merging is a widely adopted technique in vision transformers (ViTs) to reduce computation with minimal accuracy degradation. However, in tasks that require fine-grained outputs, such as text-unified object recognition, aggressive merging reaches a limit due to accuracy loss. To address this, we introduce a method that exploits residual token similarity beyond the merging threshold using difference computing, which significantly improves energy efficiency. The proposed processor integrates the post merging residual difference (PMRD) algorithm, which first reduces 34% of tokens through merging, then applies difference computing to the remaining tokens. This results in a 31% reduction in effective bit operations by leveraging residual similarity. To further exploit the resulting bit-level sparsity, the processor employs sparsity-separated accumulation with sign-magnitude representation (SSA-SM) and a bi-directional dynamic memory allocation (BDA) to optimize both computation and memory access. These enhancements yield an additional 29.4% computational energy savings. Overall, the system achieves a $2.22\times $ improvement in energy efficiency and delivers 7.23 TOPS/W on the MS COCO dataset for text-unified object recognition.
- 20 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 61 (2) , 750-762
Cache-PIM: An ECC-Compatible eDRAM Processing-in-Memory for Last-Level Cache With Triple-Level Error Correction
This article presents cache-processing-in-memory (PIM), an error correction code (ECC)-compatible embedded dynamic random access memory (eDRAM) PIM-based last-level cache (LLC) with a novel triple-level error correction. Integrating PIM into the cache system causes the existing ECC to become a performance bottleneck, leading to higher latency and decreased computational accuracy. An ECC-compatible eDRAM-PIM enables reliable in-memory computing (IMC) even in less stable DRAM environments while reducing ECC latency for PIM tasks. Cache-PIM proposes three key features: 1) triggered error correction with concurrent error detection (TECCED) reduces cell error correction latency for PIM tasks; 2) adaptive error canceling (AEC) corrects computation errors; and 3) resolution-aware single-cycle voting (RSV) reduces analog-to-digital converter (ADC) readout error. Cache-PIM is fabricated in 28-nm CMOS technology and occupies 0.66-mm2 die area. A demonstration of ViT-Base on the ImageNet dataset achieves 61% latency reduction compared to the conventional ECC and 77.1% accuracy.
- 20 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 61 (2) , 750-762
Cache-PIM: An ECC-Compatible eDRAM Processing-in-Memory for Last-Level Cache With Triple-Level Error Correction
This article presents cache-processing-in-memory (PIM), an error correction code (ECC)-compatible embedded dynamic random access memory (eDRAM) PIM-based last-level cache (LLC) with a novel triple-level error correction. Integrating PIM into the cache system causes the existing ECC to become a performance bottleneck, leading to higher latency and decreased computational accuracy. An ECC-compatible eDRAM-PIM enables reliable in-memory computing (IMC) even in less stable DRAM environments while reducing ECC latency for PIM tasks. Cache-PIM proposes three key features: 1) triggered error correction with concurrent error detection (TECCED) reduces cell error correction latency for PIM tasks; 2) adaptive error canceling (AEC) corrects computation errors; and 3) resolution-aware single-cycle voting (RSV) reduces analog-to-digital converter (ADC) readout error. Cache-PIM is fabricated in 28-nm CMOS technology and occupies 0.66-mm2 die area. A demonstration of ViT-Base on the ImageNet dataset achieves 61% latency reduction compared to the conventional ECC and 77.1% accuracy.
- 15 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 60 (10) , 3802-3815
C-Transformer: An Energy-Efficient Homogeneous DNN-Transformer/SNN-Transformer Processor for Large Language Models
In this article, we propose a new language model processor named the C-Transformer to address the external memory bottleneck of language models. It consists of three key functional blocks: 1) homogeneous deep-neural-network (DNN)–Transformer/spiking-Transformer core (HDSC) with a hybrid multiplication/accumulation unit (HMAU) to enhance hardware utilization; 2) output spike speculation unit (OSSU) to increase the energy efficiency of spike-domain processing; and 3) implicit weight generation unit (IWGU) with extended sign compression (ESC) to eliminate the external memory bottleneck. The chip is fabricated in Samsung’s 28-nm 1P8M CMOS technology and operates at a supply voltage of 0.7–1.1 V with a maximum frequency of 200 MHz and supports various tasks such as language modeling, translation, and summarization. For models like generative pre-trained transformer 2 (GPT-2), multilingual text-to-text transfer transformer (mT5), text-to-text transfer transformer (T5), and FairSeq MachineTranslation (FSMT), the C-Transformer achieves $0.21{\times }$ – $0.33{\times }$ computation energy and $0.37{\times }$ – $0.41{\times }$ external memory access (EMA) energy compared to the baseline. Our chip demonstrates 13.6% lower energy consumption than the previous state-of-the-art, despite having $2.1{\times }$ more parameters. Moreover, it consumes 63.8% less energy with a similar parameter size. The C-Transformer can complete various language model tasks with <1 s latency, notably FSMT in 0.09 s and GPT-2 in 0.656 s. By combining DNN-Transformer and Spiking-Transformer architectures, the C-Transformer enhances computational energy efficiency, eliminates external memory bottlenecks, and enables language models such as GPT-2 to achieve state-of-the-art performance on mobile devices.
- 15 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 60 (10) , 3802-3815
C-Transformer: An Energy-Efficient Homogeneous DNN-Transformer/SNN-Transformer Processor for Large Language Models
In this article, we propose a new language model processor named the C-Transformer to address the external memory bottleneck of language models. It consists of three key functional blocks: 1) homogeneous deep-neural-network (DNN)–Transformer/spiking-Transformer core (HDSC) with a hybrid multiplication/accumulation unit (HMAU) to enhance hardware utilization; 2) output spike speculation unit (OSSU) to increase the energy efficiency of spike-domain processing; and 3) implicit weight generation unit (IWGU) with extended sign compression (ESC) to eliminate the external memory bottleneck. The chip is fabricated in Samsung’s 28-nm 1P8M CMOS technology and operates at a supply voltage of 0.7–1.1 V with a maximum frequency of 200 MHz and supports various tasks such as language modeling, translation, and summarization. For models like generative pre-trained transformer 2 (GPT-2), multilingual text-to-text transfer transformer (mT5), text-to-text transfer transformer (T5), and FairSeq MachineTranslation (FSMT), the C-Transformer achieves $0.21{\times }$ – $0.33{\times }$ computation energy and $0.37{\times }$ – $0.41{\times }$ external memory access (EMA) energy compared to the baseline. Our chip demonstrates 13.6% lower energy consumption than the previous state-of-the-art, despite having $2.1{\times }$ more parameters. Moreover, it consumes 63.8% less energy with a similar parameter size. The C-Transformer can complete various language model tasks with <1 s latency, notably FSMT in 0.09 s and GPT-2 in 0.656 s. By combining DNN-Transformer and Spiking-Transformer architectures, the C-Transformer enhances computational energy efficiency, eliminates external memory bottlenecks, and enables language models such as GPT-2 to achieve state-of-the-art performance on mobile devices.
- 8 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 15 (2) , 231-243
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
As large language models (LLMs) continue to demonstrate exceptional capabilities across various domains, the challenge of achieving energy-efficient and accurate inference becomes increasingly critical. This work presents LightRot, a lightweight rotation scheme and dedicated hardware accelerator designed for low-bit LLM inference. The proposed architecture integrates Grouped Local Rotation (GLR) and Outlier Direction Aligning (ODA) algorithms with a hierarchical Fast Hadamard Transform (FHT)-based rotation unit to address key challenges in low-bit quantization, including the energy overhead of rotation operations. The proposed accelerator, implemented in a 28nm CMOS process, achieves a peak energy efficiency of 27.4 TOPS/W for 4-bit inference, surpassing prior state-of-the-art designs. Unlike conventional approaches that rely on higher-precision inference or evaluate on basic language modeling tasks like GPT-2, LightRot is optimized for advanced models such as LLaMA2-13B and LLaMA3-8B. Its performance is further validated on MT-Bench, demonstrating robust applicability to real-world conversational scenarios and redefining benchmarks for chat-based AI systems. By synergizing algorithmic innovations and hardware efficiency, this work sets a new paradigm for scalable, low-bit LLM inference, paving the way for sustainable AI advancements.
- 8 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Vol. 15 (2) , 231-243
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
As large language models (LLMs) continue to demonstrate exceptional capabilities across various domains, the challenge of achieving energy-efficient and accurate inference becomes increasingly critical. This work presents LightRot, a lightweight rotation scheme and dedicated hardware accelerator designed for low-bit LLM inference. The proposed architecture integrates Grouped Local Rotation (GLR) and Outlier Direction Aligning (ODA) algorithms with a hierarchical Fast Hadamard Transform (FHT)-based rotation unit to address key challenges in low-bit quantization, including the energy overhead of rotation operations. The proposed accelerator, implemented in a 28nm CMOS process, achieves a peak energy efficiency of 27.4 TOPS/W for 4-bit inference, surpassing prior state-of-the-art designs. Unlike conventional approaches that rely on higher-precision inference or evaluate on basic language modeling tasks like GPT-2, LightRot is optimized for advanced models such as LLaMA2-13B and LLaMA3-8B. Its performance is further validated on MT-Bench, demonstrating robust applicability to real-world conversational scenarios and redefining benchmarks for chat-based AI systems. By synergizing algorithmic innovations and hardware efficiency, this work sets a new paradigm for scalable, low-bit LLM inference, paving the way for sustainable AI advancements.