JHJeongmin Hong

Jeongmin Hong

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jeongmin Hong has a total of 12 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Non-Invasive, Memory Access-Triggered Near-Data Processing for DNN Training Acceleration on GPUs

    by Hyungkyu Ham, Hyunuk Cho, Minjae Kim, Jueon Park, Jeongmin Hong, Hyojin Sung, Eunhyeok Park, Euicheol Lim, Gwangsun Kim

    To address the off-chip bandwidth (BW) and memory capacity challenges of GPUs in DNN training, we propose a memory access-triggered near-data processing mat NDP architecture that offloads memory- and communication-bound operations. With mat NDP, normal memory accesses also serve as implicit NDP requests to enable NDP in a non-invasive manner without modifying core-side ISA/microarchitecture/SW, for practical NDP. In addition, mat NDP enables on-the-fly NDP where the data already supplied in normal memory requests for compute-bound operations are also simultaneously used for NDP; thus, matNDP can overlap even dependent kernels while also reducing memory traffic. Moreover, with the overlap, memory bandwidth (BW) underutilized by GPU cores can be used by NDP units to improve performance even under the same total memory BW. The matNDP units can be deployed to heterogeneous memory devices in a system. First, we deploy them near GPU’s memory controllers. Secondly, our NDP units are deployed in memory expanders connected to multiple GPUs to create an NDP-enabled memory eXpander Network (NDPXNet) . It can entirely offload gradient reduction and optimizer in data-parallel training, achieving additional speedups while eliminating redundancy in memory usage and optimizer execution. As a result, we 1) enable NDP without core HW/SW changes, 2) overlap the execution of dependent layers, and 3) offload both memory- and communicationbound operations from GPUs in DNN training. Through our deep learning compiler support, NDP kernels can be generated automatically without any model code modification. Consequently, mat NDP can improve training throughput by up to 2.73× and reduce energy by up to 41.4%.


  • Overcoming Memory Capacity Wall of GPUs With Heterogeneous Memory Stack

    by Jeongmin Hong, Sungjun Cho, Gwangsun Kim

    We propose to overcome the memory capacity limitation of GPUs with a Heterogeneous Memory Stack (HMS) that integrates Storage Class Memory (SCM) and DRAM in a 3D memory stack. By effectively utilizing the DRAM as a cache, the HMS addresses the latency, bandwidth, and energy challenges of the SCM. For workloads that mandate memory oversubscription, the HMS can capture a larger fraction of the memory footprint compared to an HBM, providing a significant performance uplift. To enable a performant DRAM cache architecture, we propose a DRAM cache bypass mechanism that avoids DRAM cache thrashing by 100,000 s of GPU threads. Furthermore, we propose a Tag Cache which uses part of L2 cache to hold tags of DRAM cachelines and reduce DRAM cache probe traffic, while enabling adjustment of L2 cache capacity used for Tag Cache. In addition, our DRAM cache organization enables fetching all DRAM cacheline tags in a row with a single column access, avoiding excessive DRAM cache probe traffic from Tag Cache misses. Our results show that the HMS outperforms the baseline HBM system under memory oversubscription by up to 9.8× (2.8× on average).


  • Runtime Support for Accelerating CNN Models on Digital DRAM Processing-in-Memory Hardware

    by Yongwon Shin, Juseong Park, Jeongmin Hong, Hyojin Sung

    Processing-in-memory (PIM) provides promising solutions to the main memory bottleneck by placing computational logic in or near memory devices to reduce data movement overheads. Recent work explored how commercial DRAM can feature digital PIM logic while meeting fab-level energy and area constraints, and showed a significant speedup in the inference time of data-intensive deep learning models. However, convolutional neural network (CNN) models were not considered as main targets for the commercial DRAM-PIM due to their compute-intensive convolution layers. Moreover, recent studies revealed that the area and power constraints on memory die prevent DRAM-PIM from competing with GPUs and specialized accelerators in accelerating them. Recently, mobile CNN models have increasingly adopted a composition of depthwise and pointwise convolutions instead of such compute-intensive convolutions to reduce computation cost without accuracy drop. In this paper, we show that 1x1 convolution can be offloaded for PIM acceleration with integrated runtime support and without any hardware or algorithm changes. We provide further speedup with parallel execution on GPU and DRAM-PIM and code generation optimizations. Our solution achieves up to 35.2% (31.6% on average) speedup for all 1x1 convolutions for mobile CNN models against GPU.


  • Near-Data Processing in Memory Expander for DNN Acceleration on GPUs

    by Hyungkyu Ham, Hyunuk Cho, Minjae Kim, Jueon Park, Jeongmin Hong, Hyojin Sung, Eunhyeok Park, Euicheol Lim, Gwangsun Kim

    We propose a near-data processing (NDP) architecture that exploits a memory expander with byte-addressable memory-semantic interconnect to accelerate memory-bound operations in deep neural networks (DNNs). Our architecture can execute NDP operations on the memory traffic from the GPU on-the-fly by employing bump-in-the-wire NDP logic between the off-chip link and memory controller. In addition, the memory-bound operations executed on the NDP unit can be effectively overlapped with compute-intensive operations executed on a GPU, even if the two operations have a dependency. Furthermore, the NDP offloading can be automatically done by the compiler without any code modification by deep learning practitioners. Our approach can achieve a 51% speedup for training VGG-16 with batch normalization.