YOYunho Oh

Yunho Oh

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Yunho Oh has a total of 64 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Towards Segmentation-Based Address Translation for LLM Inference

    by Youngjoon Cheon, Yunho Oh, Jeongseob Ahn

    1As large language models (LLMs) continue to scale in both model size and context length, the efficiency of address translation has become a critical performance bottleneck during inference. Although LLM inference accesses model weights in a layer-wise pattern with substantial reuse across token generations, the expanding KV-cache footprint increasingly contends for TLB capacity, leaving fewer entries to cache weight translations. In this paper, we revisit segmentation-based address translation for model weights to bypass conventional TLB-based translation. Since model weights are read-only and not reclaimed for the lifetime of a serving instance, the segmentation-based address translation is well-suited to this access pattern. We place model weights in a virtually contiguous region backed by a physically contiguous memory, enabling segment-based translation that minimizes address translation overhead. Our evaluation shows that the proposed design improves overall IPC by 2.51× over the paging baseline by achieving near-zero translation overhead for model weights.

  • EONSim: An NPU Simulator for On-Chip Memory and Embedding Vector Operations

    by Sangun Choi, Jongmin Kim, Jaeyong Park, Yunho Oh

    Embedding vector operations account for a growing share of modern deep neural network workloads and exhibit input-dependent and highly skewed memory access behavior. This behavior arises from runtime index sequences, in contrast to the static loop structure of matrix operation-centric kernels. Existing neural processing unit (NPU) simulators primarily target matrix operations and derive memory accesses from tiling and scheduling logic, which limits fidelity for embedding-heavy workloads. Also, coarse on-chip memory abstractions based on aggregate bandwidth and transfer latency hinder analysis of reuse behavior and both software- and hardware-managed on-chip memory techniques. This paper presents EONSim, an embedding-aware NPU simulator that captures index-driven memory behavior. EONSim represents embedding workloads through reusable, hardware-agnostic index traces and integrates a detailed on-chip memory model that supports diverse man agement policies, including software prefetching and hardware caching. Across multiple NPU configurations, EONSim reports an average inference time error of 1.3% and an average on-chip memory access count error of 5.5%.

  • Bauhaus: Restructuring Vector Database for LLM Retrieval on CXL-Based Tiered Memory

    by Kyungbin Kim, Sungsu Ahn, Wonjung Jeong, Jongmin Kim, Sangun Choi, Minseong Gil, Minseong Kim, Dongha Jung, Yunjay Hong, Haekang Jung, et al.

    Retrieval-augmented generation pipelines store large volumes of embedding vectors in vector databases for semantic search. In Compute Express Link (CXL)-based tiered memory systems, page-level placement often promotes pages containing both hot and cold vectors, leaving many hot vectors in the long-latency, lower-bandwidth CXL tier. This imbalance increases memory access time, reduces throughput, and worsens tail latency. This paper presents Bauhaus, a software-based memory management technique that increases the share of hot vectors in host memory, thereby improving access efficiency, reducing backend stalls, and lowering tail latency. Bauhaus monitors and reorders embedding vectors at vector-level granularity using Processor Event-Based Sampling (PEBS) and Virtual Memory Area (VMA) metadata, clustering hot vectors into contiguous pages to raise hot-page density and enhance the effectiveness of page promotion. Bauhaus, integrated into the Heterogeneous Memory Software Development Kit (HMSDK) developed and publicly released by SK hynix, is evaluated on a production-grade CXL platform. Across various embedding models, datasets, and CXL-to-host memory ratios, Bauhaus achieves 42.0% higher throughput over the baseline and 15.8% over HMSDK. Bauhaus sustains high RAG performance under constrained host memory capacity, offering a practical solution for large-scale vector database deployments in modern datacenter environments.


  • Open Access

    BiKD: Bidirectional Kernel Decomposition for Large-Scale GCNs on GPU

    by Inje Kim, Jihun Lee, Jong Hyun Jeong, Geonwoo Choi, Myung Kuk Yoon, Yunho Oh, Gunjae Koo

  • Open Access

    TM-Training: An Energy-Efficient Tiered Memory System for Deep Learning Training in NPUs

    by Jaeyong Park, Sangun Choi, Jongmin Kim, GunJae Koo, Myung Kuk Yoon, Yunho Oh

    DRAM accounts for a large fraction of the total cost of ownership of memory systems in deep learning acceleration systems. To achieve sustainable scalability, tiered memory systems with denser technologies become critical. Prior work has proposed various tiered memory systems, but no matter how a system is designed, data movements between tiers consume substantial energy. In particular, as model sizes and memory capacity demands grow, the data movements between memory tiers become more frequent, posing a challenge that tiered memory systems may reduce deployment costs but suffer from low energy efficiency. If a memory system proactively places data into a tier and timely fetches it from the tier, then the excessive data movement between tiers can be mitigated. We find that a system can statically anticipate such behaviors for all pages and localities. With this insight, we propose a new DNN acceleration system called TM-Training using flash memory. TM-Training capitalizes on the repetitive nature of the same computational patterns during execution, enabling the static establishment of optimal data placement for subsequent operations. Moreover, TM-Training employs a new data-splitting scheme to enable precise memory management. Our evaluation demonstrates that TM-Training reduces inter-tier data traffic by 64% and achieves a 55% higher throughput per watt in training than prior work.


  • Open Access

    A Low-latency On-chip Cache Hierarchy for Load-to-use Stall Reduction in GPUs

    by Negin (Sadat) (Nematollahi Zadeh) Mahani, Hajar Falahati, Sina Darabi, Ahmad Javadi-Nezhad, Yunho Oh, Mohammad Sadrosadati, Hamid Sarbazi-Azad, Babak Falsafi

    Memory hierarchy in Graphics Processing Units (GPUs) is conventionally designed to provide high bandwidth rather than low latency. In particular, because of the high tolerance to load-to-use latency (i.e., the time that warps wait for data fetched by memory loads), GPU L1D caches are optimized for density, capacity, and low power with latencies that are often orders of magnitude longer than conventional CPU caches. However, there are many important classes of data-parallel applications (e.g., graph, tree, priority queue processing, and sparse deep learning applications) that benefit from lower load-to-use latency than that offered by modern GPUs due to their inherent divergence and low effective Thread-Level Parallelism (TLP). This paper introduces an innovative on-chip cache hierarchy that incorporates a decoupled L1D cache with reduced latency (LoTUS) and its management scheme. LoTUS is a minimally sized fully associative cache placed in each GPU subcore that captures the primary working set of data-parallel applications. It exploits conventional high-performance low-density SRAM cells and dramatically reduces load-to-use latency. We also propose an intelligent extension of LoTUS, called LoTUSage, which employs a lightweight learning-based model to predict the utility of caching requests in LoTUS. Evaluation results show that LoTUS and LoTUSage improve the average performance by 23.9% and 35.4% and reduce the average energy consumption by 27.8% and 38.5%, respectively, for the applications suffering from high load-to-use stalls with negligible area and power overheads.

  • Open Access

    Avant-Garde: Empowering GPUs with Scaled Numeric Formats

    by Minseong Gil, Dongho Ha, Simla Burcu Harma, Myung Kuk Yoon, Babak Falsafi, Won Woo Ro, Yunho Oh


  • MOST: Memory Oversubscription-Aware Scheduling for Tensor Migration on GPU Unified Storage

    by Junsu Kim, Jaebeom Jeon, Jaeyong Park, Sangun Choi, Minseong Gil, Seokin Hong, Gunjae Koo, Myung Kuk Yoon, Yunho Oh

    Deep Neural Network (DNN) training demands large memory capacities that exceed the limits of current GPU onboard memory. Expanding GPU memory with SSDs is a cost-effective approach. However, the low bandwidth of SSDs introduces severe performance bottlenecks in data management, particularly for Unified Virtual Memory (UVM)-based systems. The default on-demand migration mechanism in UVM causes frequent page faults and stalls, exacerbated by memory oversubscription and eviction processes along the critical path. To address these challenges, this paper proposes Memory Oversubscription-aware Scheduling for Tensor Migration (MOST), a software framework designed to improve data migration in UVM environments. MOST profiles memory access behavior and quantifies the impact of memory oversubscription stalls and schedules tensor migrations to minimize overall training time. With the profiling results, MOST executes newly designed pre-eviction and prefetching instructions within DNN kernel code. MOST effectively selects and migrates tensors that can mitigate memory oversubscription stalls, thus reducing training time. Our evaluation shows that MOST achieves an average speedup of 22.9% and 12.8% over state-of-the-art techniques, DeepUM and G10, respectively.


  • Open Access

    SSFFT: Energy-Efficient Selective Scaling for Fast Fourier Transform in Embedded GPUs

    by Dongwon Yang, Jaebeom Jeon, Minseong Gil, Junsu Kim, Seondeok Kim, Gunjae Koo, Myung Kuk Yoon, Yunho Oh