Yunho Oh
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Yunho Oh has a total of 64 co-authors. Here are the top co-authors by number of shared publications:
- Myung Kuk Yoon
Department of Computer Science and Engineering, Ewha Womans University, Seoul, South Korea
14 shared publications - Gunjae Koo
Korea University, Seoul, Republic of Korea
10 shared publications - Sangun Choi
School of Electrical Engineering, Korea University, Seoul, South Korea
7 shared publications - Won Woo Ro
Yonsei University
7 shared publications - Jaebeom Jeon
Korea University, Seoul, Republic of Korea
6 shared publications - Babak Falsafi
EPFL
5 shared publications
Latest Publications
- 15 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 190-193
Towards Segmentation-Based Address Translation for LLM Inference
1As large language models (LLMs) continue to scale in both model size and context length, the efficiency of address translation has become a critical performance bottleneck during inference. Although LLM inference accesses model weights in a layer-wise pattern with substantial reuse across token generations, the expanding KV-cache footprint increasingly contends for TLB capacity, leaving fewer entries to cache weight translations. In this paper, we revisit segmentation-based address translation for model weights to bypass conventional TLB-based translation. Since model weights are read-only and not reclaimed for the lifetime of a serving instance, the segmentation-based address translation is well-suited to this access pattern. We place model weights in a virtually contiguous region backed by a physically contiguous memory, enabling segment-based translation that minimizes address translation overhead. Our evaluation shows that the proposed design improves overall IPC by 2.51× over the paging baseline by achieving near-zero translation overhead for model weights.
- 15 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 190-193
Towards Segmentation-Based Address Translation for LLM Inference
1As large language models (LLMs) continue to scale in both model size and context length, the efficiency of address translation has become a critical performance bottleneck during inference. Although LLM inference accesses model weights in a layer-wise pattern with substantial reuse across token generations, the expanding KV-cache footprint increasingly contends for TLB capacity, leaving fewer entries to cache weight translations. In this paper, we revisit segmentation-based address translation for model weights to bypass conventional TLB-based translation. Since model weights are read-only and not reclaimed for the lifetime of a serving instance, the segmentation-based address translation is well-suited to this access pattern. We place model weights in a virtually contiguous region backed by a physically contiguous memory, enabling segment-based translation that minimizes address translation overhead. Our evaluation shows that the proposed design improves overall IPC by 2.51× over the paging baseline by achieving near-zero translation overhead for model weights.
- 24 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 73-76
EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations
Embedding vector operations account for a growing share of modern deep neural network workloads and exhibit input-dependent and highly skewed memory access behavior. This behavior arises from runtime index sequences, in contrast to the static loop structure of matrix operation-centric kernels. Existing neural processing unit (NPU) simulators primarily target matrix operations and derive memory accesses from tiling and scheduling logic, which limits fidelity for embedding-heavy workloads. Also, coarse on-chip memory abstractions based on aggregate bandwidth and transfer latency hinder analysis of reuse behavior and both software- and hardware-managed on-chip memory techniques. This paper presents EONSim, an embedding-aware NPU simulator that captures index-driven memory behavior. EONSim represents embedding workloads through reusable, hardware-agnostic index traces and integrates a detailed on-chip memory model that supports diverse man agement policies, including software prefetching and hardware caching. Across multiple NPU configurations, EONSim reports an average inference time error of 1.3% and an average on-chip memory access count error of 5.5%.
- 24 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 25 (1) , 73-76
EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations
Embedding vector operations account for a growing share of modern deep neural network workloads and exhibit input-dependent and highly skewed memory access behavior. This behavior arises from runtime index sequences, in contrast to the static loop structure of matrix operation-centric kernels. Existing neural processing unit (NPU) simulators primarily target matrix operations and derive memory accesses from tiling and scheduling logic, which limits fidelity for embedding-heavy workloads. Also, coarse on-chip memory abstractions based on aggregate bandwidth and transfer latency hinder analysis of reuse behavior and both software- and hardware-managed on-chip memory techniques. This paper presents EONSim, an embedding-aware NPU simulator that captures index-driven memory behavior. EONSim represents embedding workloads through reusable, hardware-agnostic index traces and integrates a detailed on-chip memory model that supports diverse man agement policies, including software prefetching and hardware caching. Across multiple NPU configurations, EONSim reports an average inference time error of 1.3% and an average on-chip memory access count error of 5.5%.
- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (4) , 1309-1322
Bauhaus: Restructuring Vector Database for LLM Retrieval on CXL-Based Tiered Memory
Retrieval-augmented generation pipelines store large volumes of embedding vectors in vector databases for semantic search. In Compute Express Link (CXL)-based tiered memory systems, page-level placement often promotes pages containing both hot and cold vectors, leaving many hot vectors in the long-latency, lower-bandwidth CXL tier. This imbalance increases memory access time, reduces throughput, and worsens tail latency. This paper presents Bauhaus, a software-based memory management technique that increases the share of hot vectors in host memory, thereby improving access efficiency, reducing backend stalls, and lowering tail latency. Bauhaus monitors and reorders embedding vectors at vector-level granularity using Processor Event-Based Sampling (PEBS) and Virtual Memory Area (VMA) metadata, clustering hot vectors into contiguous pages to raise hot-page density and enhance the effectiveness of page promotion. Bauhaus, integrated into the Heterogeneous Memory Software Development Kit (HMSDK) developed and publicly released by SK hynix, is evaluated on a production-grade CXL platform. Across various embedding models, datasets, and CXL-to-host memory ratios, Bauhaus achieves 42.0% higher throughput over the baseline and 15.8% over HMSDK. Bauhaus sustains high RAG performance under constrained host memory capacity, offering a practical solution for large-scale vector database deployments in modern datacenter environments.
- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (4) , 1309-1322
Bauhaus: Restructuring Vector Database for LLM Retrieval on CXL-Based Tiered Memory
Retrieval-augmented generation pipelines store large volumes of embedding vectors in vector databases for semantic search. In Compute Express Link (CXL)-based tiered memory systems, page-level placement often promotes pages containing both hot and cold vectors, leaving many hot vectors in the long-latency, lower-bandwidth CXL tier. This imbalance increases memory access time, reduces throughput, and worsens tail latency. This paper presents Bauhaus, a software-based memory management technique that increases the share of hot vectors in host memory, thereby improving access efficiency, reducing backend stalls, and lowering tail latency. Bauhaus monitors and reorders embedding vectors at vector-level granularity using Processor Event-Based Sampling (PEBS) and Virtual Memory Area (VMA) metadata, clustering hot vectors into contiguous pages to raise hot-page density and enhance the effectiveness of page promotion. Bauhaus, integrated into the Heterogeneous Memory Software Development Kit (HMSDK) developed and publicly released by SK hynix, is evaluated on a production-grade CXL platform. Across various embedding models, datasets, and CXL-to-host memory ratios, Bauhaus achieves 42.0% higher throughput over the baseline and 15.8% over HMSDK. Bauhaus sustains high RAG performance under constrained host memory capacity, offering a practical solution for large-scale vector database deployments in modern datacenter environments.
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- p. 1
BiKD: Bidirectional Kernel Decomposition for Large-Scale GCNs on GPU
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- p. 1
Open AccessBiKD: Bidirectional Kernel Decomposition for Large-Scale GCNs on GPU
- Open Access
- 3 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 21 (4) , 1-26
TM-Training: An Energy-Efficient Tiered Memory System for Deep Learning Training in NPUs
DRAM accounts for a large fraction of the total cost of ownership of memory systems in deep learning acceleration systems. To achieve sustainable scalability, tiered memory systems with denser technologies become critical. Prior work has proposed various tiered memory systems, but no matter how a system is designed, data movements between tiers consume substantial energy. In particular, as model sizes and memory capacity demands grow, the data movements between memory tiers become more frequent, posing a challenge that tiered memory systems may reduce deployment costs but suffer from low energy efficiency. If a memory system proactively places data into a tier and timely fetches it from the tier, then the excessive data movement between tiers can be mitigated. We find that a system can statically anticipate such behaviors for all pages and localities. With this insight, we propose a new DNN acceleration system called TM-Training using flash memory. TM-Training capitalizes on the repetitive nature of the same computational patterns during execution, enabling the static establishment of optimal data placement for subsequent operations. Moreover, TM-Training employs a new data-splitting scheme to enable precise memory management. Our evaluation demonstrates that TM-Training reduces inter-tier data traffic by 64% and achieves a 55% higher throughput per watt in training than prior work.
- 3 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 21 (4) , 1-26
Open AccessTM-Training: An Energy-Efficient Tiered Memory System for Deep Learning Training in NPUs
DRAM accounts for a large fraction of the total cost of ownership of memory systems in deep learning acceleration systems. To achieve sustainable scalability, tiered memory systems with denser technologies become critical. Prior work has proposed various tiered memory systems, but no matter how a system is designed, data movements between tiers consume substantial energy. In particular, as model sizes and memory capacity demands grow, the data movements between memory tiers become more frequent, posing a challenge that tiered memory systems may reduce deployment costs but suffer from low energy efficiency. If a memory system proactively places data into a tier and timely fetches it from the tier, then the excessive data movement between tiers can be mitigated. We find that a system can statically anticipate such behaviors for all pages and localities. With this insight, we propose a new DNN acceleration system called TM-Training using flash memory. TM-Training capitalizes on the repetitive nature of the same computational patterns during execution, enabling the static establishment of optimal data placement for subsequent operations. Moreover, TM-Training employs a new data-splitting scheme to enable precise memory management. Our evaluation demonstrates that TM-Training reduces inter-tier data traffic by 64% and achieves a 55% higher throughput per watt in training than prior work.
- Open Access
- 18 August 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
A Low-latency On-chip Cache Hierarchy for Load-to-use Stall Reduction in GPUs
Memory hierarchy in Graphics Processing Units (GPUs) is conventionally designed to provide high bandwidth rather than low latency. In particular, because of the high tolerance to load-to-use latency (i.e., the time that warps wait for data fetched by memory loads), GPU L1D caches are optimized for density, capacity, and low power with latencies that are often orders of magnitude longer than conventional CPU caches. However, there are many important classes of data-parallel applications (e.g., graph, tree, priority queue processing, and sparse deep learning applications) that benefit from lower load-to-use latency than that offered by modern GPUs due to their inherent divergence and low effective Thread-Level Parallelism (TLP). This paper introduces an innovative on-chip cache hierarchy that incorporates a decoupled L1D cache with reduced latency (LoTUS) and its management scheme. LoTUS is a minimally sized fully associative cache placed in each GPU subcore that captures the primary working set of data-parallel applications. It exploits conventional high-performance low-density SRAM cells and dramatically reduces load-to-use latency. We also propose an intelligent extension of LoTUS, called LoTUSage, which employs a lightweight learning-based model to predict the utility of caching requests in LoTUS. Evaluation results show that LoTUS and LoTUSage improve the average performance by 23.9% and 35.4% and reduce the average energy consumption by 27.8% and 38.5%, respectively, for the applications suffering from high load-to-use stalls with negligible area and power overheads.
- 18 August 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
Open AccessA Low-latency On-chip Cache Hierarchy for Load-to-use Stall Reduction in GPUs
Memory hierarchy in Graphics Processing Units (GPUs) is conventionally designed to provide high bandwidth rather than low latency. In particular, because of the high tolerance to load-to-use latency (i.e., the time that warps wait for data fetched by memory loads), GPU L1D caches are optimized for density, capacity, and low power with latencies that are often orders of magnitude longer than conventional CPU caches. However, there are many important classes of data-parallel applications (e.g., graph, tree, priority queue processing, and sparse deep learning applications) that benefit from lower load-to-use latency than that offered by modern GPUs due to their inherent divergence and low effective Thread-Level Parallelism (TLP). This paper introduces an innovative on-chip cache hierarchy that incorporates a decoupled L1D cache with reduced latency (LoTUS) and its management scheme. LoTUS is a minimally sized fully associative cache placed in each GPU subcore that captures the primary working set of data-parallel applications. It exploits conventional high-performance low-density SRAM cells and dramatically reduces load-to-use latency. We also propose an intelligent extension of LoTUS, called LoTUSage, which employs a lightweight learning-based model to predict the utility of caching requests in LoTUS. Evaluation results show that LoTUS and LoTUSage improve the average performance by 23.9% and 35.4% and reduce the average energy consumption by 27.8% and 38.5%, respectively, for the applications suffering from high load-to-use stalls with negligible area and power overheads.
- Open Access
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 153-165
Avant-Garde: Empowering GPUs with Scaled Numeric Formats
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 153-165
Open AccessAvant-Garde: Empowering GPUs with Scaled Numeric Formats
- 17 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 213-216
MOST: Memory Oversubscription-Aware Scheduling for Tensor Migration on GPU Unified Storage
Deep Neural Network (DNN) training demands large memory capacities that exceed the limits of current GPU onboard memory. Expanding GPU memory with SSDs is a cost-effective approach. However, the low bandwidth of SSDs introduces severe performance bottlenecks in data management, particularly for Unified Virtual Memory (UVM)-based systems. The default on-demand migration mechanism in UVM causes frequent page faults and stalls, exacerbated by memory oversubscription and eviction processes along the critical path. To address these challenges, this paper proposes Memory Oversubscription-aware Scheduling for Tensor Migration (MOST), a software framework designed to improve data migration in UVM environments. MOST profiles memory access behavior and quantifies the impact of memory oversubscription stalls and schedules tensor migrations to minimize overall training time. With the profiling results, MOST executes newly designed pre-eviction and prefetching instructions within DNN kernel code. MOST effectively selects and migrates tensors that can mitigate memory oversubscription stalls, thus reducing training time. Our evaluation shows that MOST achieves an average speedup of 22.9% and 12.8% over state-of-the-art techniques, DeepUM and G10, respectively.
- 17 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 213-216
MOST: Memory Oversubscription-Aware Scheduling for Tensor Migration on GPU Unified Storage
Deep Neural Network (DNN) training demands large memory capacities that exceed the limits of current GPU onboard memory. Expanding GPU memory with SSDs is a cost-effective approach. However, the low bandwidth of SSDs introduces severe performance bottlenecks in data management, particularly for Unified Virtual Memory (UVM)-based systems. The default on-demand migration mechanism in UVM causes frequent page faults and stalls, exacerbated by memory oversubscription and eviction processes along the critical path. To address these challenges, this paper proposes Memory Oversubscription-aware Scheduling for Tensor Migration (MOST), a software framework designed to improve data migration in UVM environments. MOST profiles memory access behavior and quantifies the impact of memory oversubscription stalls and schedules tensor migrations to minimize overall training time. With the profiling results, MOST executes newly designed pre-eviction and prefetching instructions within DNN kernel code. MOST effectively selects and migrates tensors that can mitigate memory oversubscription stalls, thus reducing training time. Our evaluation shows that MOST achieves an average speedup of 22.9% and 12.8% over state-of-the-art techniques, DeepUM and G10, respectively.
- Open Access
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 85-96
SSFFT: Energy-Efficient Selective Scaling for Fast Fourier Transform in Embedded GPUs
- 13 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 85-96
Open AccessSSFFT: Energy-Efficient Selective Scaling for Fast Fourier Transform in Embedded GPUs