Jeongmin Hong
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Jeongmin Hong has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Gwangsun Kim
POSTECH, Pohang, Republic of Korea
3 shared publications - Hyojin Sung
Seoul National University, Seoul, Republic of Korea
3 shared publications - Euicheol Lim
SK Hynix Inc
2 shared publications - Eunhyeok Park
Pohang University of Science and Technology (POSTECH) Republic of Korea
2 shared publications - Hyungkyu Ham
POSTECH, Pohang, Republic of Korea
2 shared publications - Hyunuk Cho
Pohang University of Science and Technology, Pohang, Gyeongsangbuk-do, Republic of Korea
2 shared publications
Latest Publications
- Open Access
- 23 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 142651-142667
Non-Invasive, Memory Access-Triggered Near-Data Processing for DNN Training Acceleration on GPUs
To address the off-chip bandwidth (BW) and memory capacity challenges of GPUs in DNN training, we propose a memory access-triggered near-data processing mat NDP architecture that offloads memory- and communication-bound operations. With mat NDP, normal memory accesses also serve as implicit NDP requests to enable NDP in a non-invasive manner without modifying core-side ISA/microarchitecture/SW, for practical NDP. In addition, mat NDP enables on-the-fly NDP where the data already supplied in normal memory requests for compute-bound operations are also simultaneously used for NDP; thus, matNDP can overlap even dependent kernels while also reducing memory traffic. Moreover, with the overlap, memory bandwidth (BW) underutilized by GPU cores can be used by NDP units to improve performance even under the same total memory BW. The matNDP units can be deployed to heterogeneous memory devices in a system. First, we deploy them near GPU’s memory controllers. Secondly, our NDP units are deployed in memory expanders connected to multiple GPUs to create an NDP-enabled memory eXpander Network (NDPXNet) . It can entirely offload gradient reduction and optimizer in data-parallel training, achieving additional speedups while eliminating redundancy in memory usage and optimizer execution. As a result, we 1) enable NDP without core HW/SW changes, 2) overlap the execution of dependent layers, and 3) offload both memory- and communicationbound operations from GPUs in DNN training. Through our deep learning compiler support, NDP kernels can be generated automatically without any model code modification. Consequently, mat NDP can improve training throughput by up to 2.73× and reduce energy by up to 41.4%.
- 23 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 142651-142667
Open AccessNon-Invasive, Memory Access-Triggered Near-Data Processing for DNN Training Acceleration on GPUs
To address the off-chip bandwidth (BW) and memory capacity challenges of GPUs in DNN training, we propose a memory access-triggered near-data processing mat NDP architecture that offloads memory- and communication-bound operations. With mat NDP, normal memory accesses also serve as implicit NDP requests to enable NDP in a non-invasive manner without modifying core-side ISA/microarchitecture/SW, for practical NDP. In addition, mat NDP enables on-the-fly NDP where the data already supplied in normal memory requests for compute-bound operations are also simultaneously used for NDP; thus, matNDP can overlap even dependent kernels while also reducing memory traffic. Moreover, with the overlap, memory bandwidth (BW) underutilized by GPU cores can be used by NDP units to improve performance even under the same total memory BW. The matNDP units can be deployed to heterogeneous memory devices in a system. First, we deploy them near GPU’s memory controllers. Secondly, our NDP units are deployed in memory expanders connected to multiple GPUs to create an NDP-enabled memory eXpander Network (NDPXNet) . It can entirely offload gradient reduction and optimizer in data-parallel training, achieving additional speedups while eliminating redundancy in memory usage and optimizer execution. As a result, we 1) enable NDP without core HW/SW changes, 2) overlap the execution of dependent layers, and 3) offload both memory- and communicationbound operations from GPUs in DNN training. Through our deep learning compiler support, NDP kernels can be generated automatically without any model code modification. Consequently, mat NDP can improve training throughput by up to 2.73× and reduce energy by up to 41.4%.
- 8 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 61-64
Overcoming Memory Capacity Wall of GPUs With Heterogeneous Memory Stack
We propose to overcome the memory capacity limitation of GPUs with a Heterogeneous Memory Stack (HMS) that integrates Storage Class Memory (SCM) and DRAM in a 3D memory stack. By effectively utilizing the DRAM as a cache, the HMS addresses the latency, bandwidth, and energy challenges of the SCM. For workloads that mandate memory oversubscription, the HMS can capture a larger fraction of the memory footprint compared to an HBM, providing a significant performance uplift. To enable a performant DRAM cache architecture, we propose a DRAM cache bypass mechanism that avoids DRAM cache thrashing by 100,000 s of GPU threads. Furthermore, we propose a Tag Cache which uses part of L2 cache to hold tags of DRAM cachelines and reduce DRAM cache probe traffic, while enabling adjustment of L2 cache capacity used for Tag Cache. In addition, our DRAM cache organization enables fetching all DRAM cacheline tags in a row with a single column access, avoiding excessive DRAM cache probe traffic from Tag Cache misses. Our results show that the HMS outperforms the baseline HBM system under memory oversubscription by up to 9.8× (2.8× on average).
- 8 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 61-64
Overcoming Memory Capacity Wall of GPUs With Heterogeneous Memory Stack
We propose to overcome the memory capacity limitation of GPUs with a Heterogeneous Memory Stack (HMS) that integrates Storage Class Memory (SCM) and DRAM in a 3D memory stack. By effectively utilizing the DRAM as a cache, the HMS addresses the latency, bandwidth, and energy challenges of the SCM. For workloads that mandate memory oversubscription, the HMS can capture a larger fraction of the memory footprint compared to an HBM, providing a significant performance uplift. To enable a performant DRAM cache architecture, we propose a DRAM cache bypass mechanism that avoids DRAM cache thrashing by 100,000 s of GPU threads. Furthermore, we propose a Tag Cache which uses part of L2 cache to hold tags of DRAM cachelines and reduce DRAM cache probe traffic, while enabling adjustment of L2 cache capacity used for Tag Cache. In addition, our DRAM cache organization enables fetching all DRAM cacheline tags in a row with a single column access, avoiding excessive DRAM cache probe traffic from Tag Cache misses. Our results show that the HMS outperforms the baseline HBM system under memory oversubscription by up to 9.8× (2.8× on average).
- 13 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 33-36
Runtime Support for Accelerating CNN Models on Digital DRAM Processing-in-Memory Hardware
Processing-in-memory (PIM) provides promising solutions to the main memory bottleneck by placing computational logic in or near memory devices to reduce data movement overheads. Recent work explored how commercial DRAM can feature digital PIM logic while meeting fab-level energy and area constraints, and showed a significant speedup in the inference time of data-intensive deep learning models. However, convolutional neural network (CNN) models were not considered as main targets for the commercial DRAM-PIM due to their compute-intensive convolution layers. Moreover, recent studies revealed that the area and power constraints on memory die prevent DRAM-PIM from competing with GPUs and specialized accelerators in accelerating them. Recently, mobile CNN models have increasingly adopted a composition of depthwise and pointwise convolutions instead of such compute-intensive convolutions to reduce computation cost without accuracy drop. In this paper, we show that 1x1 convolution can be offloaded for PIM acceleration with integrated runtime support and without any hardware or algorithm changes. We provide further speedup with parallel execution on GPU and DRAM-PIM and code generation optimizations. Our solution achieves up to 35.2% (31.6% on average) speedup for all 1x1 convolutions for mobile CNN models against GPU.
- 13 June 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 33-36
Runtime Support for Accelerating CNN Models on Digital DRAM Processing-in-Memory Hardware
Processing-in-memory (PIM) provides promising solutions to the main memory bottleneck by placing computational logic in or near memory devices to reduce data movement overheads. Recent work explored how commercial DRAM can feature digital PIM logic while meeting fab-level energy and area constraints, and showed a significant speedup in the inference time of data-intensive deep learning models. However, convolutional neural network (CNN) models were not considered as main targets for the commercial DRAM-PIM due to their compute-intensive convolution layers. Moreover, recent studies revealed that the area and power constraints on memory die prevent DRAM-PIM from competing with GPUs and specialized accelerators in accelerating them. Recently, mobile CNN models have increasingly adopted a composition of depthwise and pointwise convolutions instead of such compute-intensive convolutions to reduce computation cost without accuracy drop. In this paper, we show that 1x1 convolution can be offloaded for PIM acceleration with integrated runtime support and without any hardware or algorithm changes. We provide further speedup with parallel execution on GPU and DRAM-PIM and code generation optimizations. Our solution achieves up to 35.2% (31.6% on average) speedup for all 1x1 convolutions for mobile CNN models against GPU.
- 9 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 20 (2) , 171-174
Near-Data Processing in Memory Expander for DNN Acceleration on GPUs
We propose a near-data processing (NDP) architecture that exploits a memory expander with byte-addressable memory-semantic interconnect to accelerate memory-bound operations in deep neural networks (DNNs). Our architecture can execute NDP operations on the memory traffic from the GPU on-the-fly by employing bump-in-the-wire NDP logic between the off-chip link and memory controller. In addition, the memory-bound operations executed on the NDP unit can be effectively overlapped with compute-intensive operations executed on a GPU, even if the two operations have a dependency. Furthermore, the NDP offloading can be automatically done by the compiler without any code modification by deep learning practitioners. Our approach can achieve a 51% speedup for training VGG-16 with batch normalization.
- 9 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 20 (2) , 171-174
Near-Data Processing in Memory Expander for DNN Acceleration on GPUs
We propose a near-data processing (NDP) architecture that exploits a memory expander with byte-addressable memory-semantic interconnect to accelerate memory-bound operations in deep neural networks (DNNs). Our architecture can execute NDP operations on the memory traffic from the GPU on-the-fly by employing bump-in-the-wire NDP logic between the off-chip link and memory controller. In addition, the memory-bound operations executed on the NDP unit can be effectively overlapped with compute-intensive operations executed on a GPU, even if the two operations have a dependency. Furthermore, the NDP offloading can be automatically done by the compiler without any code modification by deep learning practitioners. Our approach can achieve a 51% speedup for training VGG-16 with batch normalization.