Sungmin Yun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Sungmin Yun has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Jung Ho Ahn
Seoul National University, Seoul, Republic of Korea
6 shared publications - Eojin Lee
Inha University, Incheon, Republic of Korea
4 shared publications - Byeongho Kim
Seoul National University, Seoul, South Korea
3 shared publications - Hwayong Nam
Seoul National University
3 shared publications - Jaehyun Park
Seoul National University, Seoul, Republic of Korea
3 shared publications - Yongsuk Kwon
Samsung Electronics, Suwon, Republic of Korea
3 shared publications
Latest Publications
- Open Access
- 5 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 795-810
Clone: A Collaborative Multi-device System for Retrieval-Augmented Generation over CXL
- 5 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 795-810
Open AccessClone: A Collaborative Multi-device System for Retrieval-Augmented Generation over CXL
- 24 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 265-268
SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency
Large Language Models (LLMs) applying Mixture-of-Experts (MoE) scale to trillions of parameters but require vast memory, motivating a line of research to offload expert weights from fast-but-small DRAM (HBM) to denser Flash SSDs. While SSDs provide cost-effective capacity, their read energy per bit is substantially higher than that of DRAM. This paper quantitatively analyzes the energy implications of offloading MoE expert weights to SSDs during the critical decode stage of LLM inference. Our analysis, comparing SSD, CPU memory (DDR), and HBM storage scenarios for models like DeepSeek-R1, reveals that offloading MoE weights to current SSDs drastically increases per-token-generation energy consumption (e.g., by up to $\sim 12\times$ compared to the HBM baseline), dominating the total inference energy budget. Although techniques like prefetching effectively hide access latency, they cannot mitigate this fundamental energy penalty. We further explore future technological scaling, finding that the inherent sparsity of MoE models could potentially make SSDs energy-viable if Flash read energy improves significantly, roughly by an order of magnitude.
- 24 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 265-268
SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency
Large Language Models (LLMs) applying Mixture-of-Experts (MoE) scale to trillions of parameters but require vast memory, motivating a line of research to offload expert weights from fast-but-small DRAM (HBM) to denser Flash SSDs. While SSDs provide cost-effective capacity, their read energy per bit is substantially higher than that of DRAM. This paper quantitatively analyzes the energy implications of offloading MoE expert weights to SSDs during the critical decode stage of LLM inference. Our analysis, comparing SSD, CPU memory (DDR), and HBM storage scenarios for models like DeepSeek-R1, reveals that offloading MoE weights to current SSDs drastically increases per-token-generation energy consumption (e.g., by up to $\sim 12\times$ compared to the HBM baseline), dominating the total inference energy budget. Although techniques like prefetching effectively hide access latency, they cannot mitigate this fundamental energy penalty. We further explore future technological scaling, finding that the inherent sparsity of MoE models could potentially make SSDs energy-viable if Flash read energy improves significantly, roughly by an order of magnitude.
- 14 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (1) , 173-176
Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
Retrieval-Augmented Generation (RAG) is crucial for improving the quality of large language models by injecting proper contexts extracted from external sources. RAG requires high-throughput, low-latency Approximate Nearest Neighbor Search (ANNS) over billion-scale vector databases. Conventional DRAM/SSD solutions face capacity/latency limits, whereas specialized hardware or RDMA clusters lack flexibility or incur network overhead. We present COSMOS, integrating general-purpose cores within CXL memory devices for full ANNS offload and introducing rank-level parallel distance computation to maximize memory bandwidth. We also propose an adjacency-aware data placement that balances search loads across CXL devices based on inter-cluster proximity. Evaluations on SIFT1B and DEEP1B traces show that COSMOS achieves up to 6.72× higher throughput than the baseline CXL system and 2.35× over a state-of-the-art CXL-based solution, demonstrating scalability for RAG pipelines.
- 14 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (1) , 173-176
Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
Retrieval-Augmented Generation (RAG) is crucial for improving the quality of large language models by injecting proper contexts extracted from external sources. RAG requires high-throughput, low-latency Approximate Nearest Neighbor Search (ANNS) over billion-scale vector databases. Conventional DRAM/SSD solutions face capacity/latency limits, whereas specialized hardware or RDMA clusters lack flexibility or incur network overhead. We present COSMOS, integrating general-purpose cores within CXL memory devices for full ANNS offload and introducing rank-level parallel distance computation to maximize memory bandwidth. We also propose an adjacency-aware data placement that balances search loads across CXL devices based on inter-cluster proximity. Evaluations on SIFT1B and DEEP1B traces show that COSMOS achieves up to 6.72× higher throughput than the baseline CXL system and 2.35× over a state-of-the-art CXL-based solution, demonstrating scalability for RAG pipelines.
- Open Access
- 30 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 338-351
CLAY: CXL-based Scalable NDP Architecture Accelerating Embedding Layers
An embedding layer is one of the most critical building blocks of deep neural networks, especially for recommender systems and graph neural networks. The embedding layer dominates a large portion of the total execution time due to its large memory requirements and little data reuse in operations. To accelerate the embedding layers, dual in-line memory module (DIMM) based near-data processing architectures have been proposed. They amplify bandwidth by adding a processing unit to the DIMM’s buffer. However, prior architectures have less capacity scalability due to the limited number of memory channels. Crucially, they are limited in performance improvement due to the load imbalance problem and the limitations of DIMM-based memory systems with a multi-drop bus structure between the processing units and the host. In this paper, we propose CLAY, a CXL-based scalable near-data processing architecture that accelerates general embedding layers in DNN. Breaking away from conventional memory channel structures, CLAY interconnects the DRAM modules to reduce the data transfer overhead among DRAM modules. Furthermore, we devise a dedicated memory address mapping to mitigate load imbalance in CLAY and a packet duplication scheme that enables full utilization of CLAY by reducing the required instruction transmission bandwidth. We propose a method of scaling CLAY and a software stack to use CLAY. Compared to the state-of-the-art NDP architectures of FeaNMP and G-NMP, CLAY achieves an end-to-end speedup of up to 1.87 × and 2.77 × for recommender systems and graph neural networks, respectively.
- 30 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 338-351
Open AccessCLAY: CXL-based Scalable NDP Architecture Accelerating Embedding Layers
An embedding layer is one of the most critical building blocks of deep neural networks, especially for recommender systems and graph neural networks. The embedding layer dominates a large portion of the total execution time due to its large memory requirements and little data reuse in operations. To accelerate the embedding layers, dual in-line memory module (DIMM) based near-data processing architectures have been proposed. They amplify bandwidth by adding a processing unit to the DIMM’s buffer. However, prior architectures have less capacity scalability due to the limited number of memory channels. Crucially, they are limited in performance improvement due to the load imbalance problem and the limitations of DIMM-based memory systems with a multi-drop bus structure between the processing units and the host. In this paper, we propose CLAY, a CXL-based scalable near-data processing architecture that accelerates general embedding layers in DNN. Breaking away from conventional memory channel structures, CLAY interconnects the DRAM modules to reduce the data transfer overhead among DRAM modules. Furthermore, we devise a dedicated memory address mapping to mitigate load imbalance in CLAY and a packet duplication scheme that enables full utilization of CLAY by reducing the required instruction transmission bandwidth. We propose a method of scaling CLAY and a software stack to use CLAY. Compared to the state-of-the-art NDP architectures of FeaNMP and G-NMP, CLAY achieves an end-to-end speedup of up to 1.87 × and 2.77 × for recommender systems and graph neural networks, respectively.
- 7 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (10) , 2391-2404
GraNDe: Efficient Near-Data Processing Architecture for Graph Neural Networks
Graph Neural Network (GNN) models have attracted attention, given their high accuracy in interpreting graph data. One of the primary building blocks of a GNN model is aggregation, which gathers and averages the feature vectors corresponding to the nodes adjacent to each node. Aggregation works by multiplying the adjacency and feature matrices. The size of both matrices exceeds the on-chip cache capacity for many realistic datasets, and the adjacency matrix is highly sparse. These characteristics lead to little data reuse, causing intensive main-memory accesses during the aggregation process. Thus, aggregation exhibits memory-intensive characteristics and dominates most of the total execution time. In this paper, we propose GraNDe, an NDP architecture that accelerates memory-intensive aggregation operations by locating NDP modules near DRAM datapath to exploit rank-level parallelism. GraNDe maximizes bandwidth utilization by separating the memory channel path with the buffer chip in between so that pre-/post-processing in the host processor and reduction in NDP modules operate simultaneously. By exploring the preferred data mappings of the operand matrices to DRAM ranks, we architect GraNDe to support adaptive matrix mapping that applies the optimal mapping for each layer depending on the dimension of the layer and the configuration of a memory system. We also propose adj-bundle broadcasting and re-tiling optimizations to reduce the transfer time for adjacency matrix data and to improve feature vector data reusability by exploiting tiling with consideration of adjacency between nodes. GraNDe achieves 3.01× and 1.69× on average, and up to 4.00× and 1.98× speedups of GCN aggregation over the baseline system and the state-of-the-art NDP architecture for GCN, respectively.
- 7 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (10) , 2391-2404
GraNDe: Efficient Near-Data Processing Architecture for Graph Neural Networks
Graph Neural Network (GNN) models have attracted attention, given their high accuracy in interpreting graph data. One of the primary building blocks of a GNN model is aggregation, which gathers and averages the feature vectors corresponding to the nodes adjacent to each node. Aggregation works by multiplying the adjacency and feature matrices. The size of both matrices exceeds the on-chip cache capacity for many realistic datasets, and the adjacency matrix is highly sparse. These characteristics lead to little data reuse, causing intensive main-memory accesses during the aggregation process. Thus, aggregation exhibits memory-intensive characteristics and dominates most of the total execution time. In this paper, we propose GraNDe, an NDP architecture that accelerates memory-intensive aggregation operations by locating NDP modules near DRAM datapath to exploit rank-level parallelism. GraNDe maximizes bandwidth utilization by separating the memory channel path with the buffer chip in between so that pre-/post-processing in the host processor and reduction in NDP modules operate simultaneously. By exploring the preferred data mappings of the operand matrices to DRAM ranks, we architect GraNDe to support adaptive matrix mapping that applies the optimal mapping for each layer depending on the dimension of the layer and the configuration of a memory system. We also propose adj-bundle broadcasting and re-tiling optimizations to reduce the transfer time for adjacency matrix data and to improve feature vector data reusability by exploiting tiling with consideration of adjacency between nodes. GraNDe achieves 3.01× and 1.69× on average, and up to 4.00× and 1.98× speedups of GCN aggregation over the baseline system and the state-of-the-art NDP architecture for GCN, respectively.
- 13 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 45-48
GraNDe: Near-Data Processing Architecture With Adaptive Matrix Mapping for Graph Convolutional Networks
Graph Convolutional Network (GCN) models have attracted attention given their high accuracy in interpreting graph data. One of the primary building blocks of a GCN model is aggregation, which gathers and averages the feature vectors corresponding to the vertices adjacent to each individual vertex. Aggregation works by multiplying the adjacency and feature matrices. The size of both matrices exceeds the on-chip cache capacity, and the adjacency matrix is highly sparse. These lead to little data reuse and cause numerous main-memory accesses during the aggregation process. Thus, aggregation exhibits memory-intensive characteristics. We propose GraNDe, an NDP architecture that accelerates memory-intensive aggregation operations by locating processing elements near the DRAM datapath to exploit rank-level parallelism. By exploring the data mapping of the operand matrices to DRAM ranks, we discover that the optimal mapping differs depending on the configuration of a specific GCN layer. With our optimal layer-by-layer mapping scheme, GraNDe shows a speedup up to 4.3× compared to the baseline system on open-graph benchmark datasets.
- 13 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 45-48
GraNDe: Near-Data Processing Architecture With Adaptive Matrix Mapping for Graph Convolutional Networks
Graph Convolutional Network (GCN) models have attracted attention given their high accuracy in interpreting graph data. One of the primary building blocks of a GCN model is aggregation, which gathers and averages the feature vectors corresponding to the vertices adjacent to each individual vertex. Aggregation works by multiplying the adjacency and feature matrices. The size of both matrices exceeds the on-chip cache capacity, and the adjacency matrix is highly sparse. These lead to little data reuse and cause numerous main-memory accesses during the aggregation process. Thus, aggregation exhibits memory-intensive characteristics. We propose GraNDe, an NDP architecture that accelerates memory-intensive aggregation operations by locating processing elements near the DRAM datapath to exploit rank-level parallelism. By exploring the data mapping of the operand matrices to DRAM ranks, we discover that the optimal mapping differs depending on the configuration of a specific GCN layer. With our optimal layer-by-layer mapping scheme, GraNDe shows a speedup up to 4.3× compared to the baseline system on open-graph benchmark datasets.