JAJeongseob Ahn

Jeongseob Ahn

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jeongseob Ahn has a total of 42 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Towards Segmentation-Based Address Translation for LLM Inference

    by Youngjoon Cheon, Yunho Oh, Jeongseob Ahn

    1As large language models (LLMs) continue to scale in both model size and context length, the efficiency of address translation has become a critical performance bottleneck during inference. Although LLM inference accesses model weights in a layer-wise pattern with substantial reuse across token generations, the expanding KV-cache footprint increasingly contends for TLB capacity, leaving fewer entries to cache weight translations. In this paper, we revisit segmentation-based address translation for model weights to bypass conventional TLB-based translation. Since model weights are read-only and not reclaimed for the lifetime of a serving instance, the segmentation-based address translation is well-suited to this access pattern. We place model weights in a virtually contiguous region backed by a physically contiguous memory, enabling segment-based translation that minimizes address translation overhead. Our evaluation shows that the proposed design improves overall IPC by 2.51× over the paging baseline by achieving near-zero translation overhead for model weights.

  • Open Access

    An Efficient DNN Model Serving System using Layer-wise Caching and Direct-Host-Access

    by Jinwoo Jeong, Jeongseob Ahn

    With the increasing demand to utilize deep neural networks (DNNs) in online services, it is important to serve DNN models on GPUs in a cost-effective manner. Once the required DNN model is ready in the GPU memory, we can immediately serve the inference requests with low latency. Otherwise, it needs to load the model from host to GPU, adding a significant delay to inference. This article proposes Ignite to minimize cold-start latency while provisioning DL models from host to GPU in server environments. First, we propose LCache to effectively utilize the limited GPU memory for model serving with unique cache replacement policies. We devise layer-wise cache replacement policies tailored for executing DL inferences in a pipelined way. Second, we take advantage of the direct-host-access facility provided by commodity GPUs, allowing access to particular layers of models in the host memory directly from GPU without loading. We show that Ignite can effectively reduce the cold-start latency while increasing the throughput of serving DNN models. When deploying multiple ResNet, BERT, and RoBERTa instances on a DL inference serving system, Ignite shows a significant performance improvement compared to the pipelining technique and stable 99% tail latency.


  • GPU-Centric Memory Tiering for LLM Serving With NVIDIA Grace Hopper Superchip

    by Woohyung Choi, Jinwoo Jeong, Hanhwi Jang, Jeongseob Ahn

    This study investigates the performance of serving large language models (LLMs) with a focus on the highbandwidth interconnect between GPU and CPU using a real NVIDIA Grace Hopper Superchip. This architecture features a GPU-centric memory tiering system, comprising a performance tier with GPU memory and a capacity tier with host memory. We revisit a conventional pipelined execution for LLM inference, utilizing host memory connected via NVLink alongside GPU memory. For the Llama-3.1 8B base (FP16) model, such a GPU-centric tiered memory system meets the target latency requirements for both prefill and decoding while improving throughput compared to the in-memory case, where all model weights are maintained in GPU memory. However, even with NVLink-connected CPU memory, meeting latency constraints for large models like the 70B and 405B FP16 models remains challenging. To address this, we explore the efficacy of model quantization (e.g., AWQ) along with the pipelined execution. Our evaluation reveals that the model quantization makes the pipelined execution a viable solution for serving large models. For the Llama-3.1 70B and 405B AWQ models, we show that the pipelined execution achieves 1.6× and 2.9× throughput improvement, respectively, compared to the inmemory only case, while meeting the latency constraint.


  • Accelerating Page Migrations in Operating Systems With Intel DSA

    by Jongho Baik, Jonghyeon Kim, Chang Hyun Park, Jeongseob Ahn

    Modern server-class CPUs are introducing specialpurpose accelerators on the same chip to improve performance and efficiency for data-intensive applications. This paper presents a case for accelerating data migrations in operating systems with the Data Streaming Accelerator (DSA), a new feature by Intel. To the best of our knowledge, this is the first study that exploits a hardware-assisted data migration scheme in the operating system. We identify which Linux kernel components can benefit from the hardware acceleration, particularly focusing on the kernel subsystems that rely on the migrate_pages() kernel function. As the hardware accelerator is not suitable for transferring a small amount of data due to the HW setup overhead, this preliminary study concentrates on the design and implementation of accelerating migrate_pages() with DSA. We prototype a DSA-enabled Linux kernel and evaluate its effectiveness through two benchmarks demonstrating real-world page compaction (kcompactd) and promotion (kdamond) scenarios. In both cases, our prototype demonstrates improved throughput in page migration, benefiting both the kernel subsystem and applications.


  • Open Access

    Fast and Efficient Model Serving Using Multi-GPUs with Direct-Host-Access

    by Jinwoo Jeong, Seungsu Baek, Jeongseob Ahn

    As deep learning (DL) inference has been widely adopted for building user-facing applications in many domains, it is increasingly important for DL inference servers to achieve high throughput while preserving bounded latency. DL inference requests can be immediately served if the corresponding model is already in the GPU memory. Otherwise, it needs to load the model from host to GPU, adding a significant delay to inference. This paper proposes DeepPlan to minimize inference latency while provisioning DL models from host to GPU in server environments. First, we take advantage of the direct-host-access facility provided by commodity GPUs, allowing access to particular layers of models in the host memory directly from GPU without loading. Second, we parallelize model transmission across multiple GPUs to reduce the time for loading models from host to GPU. We show that a single inference can achieve a 1.94× speedup compared with the state-of-the-art pipelining approach for BERT-Base. When deploying multiple BERT, RoBERTa, and GPT-2 instances on a DL inference serving system, DeepPlan shows a significant performance improvement compared to the pipelining technique and stable 99% tail latency.


  • Rethinking DRAM's Page Mode With STT-MRAM

    by Byoungchan Oh, Nilmini Abeyratne, Nam Sung Kim, Jeongseob Ahn, Ronald G. Dreslinski, Trevor Mudge

    STT-MRAM is a promising candidate for drop-in replacement for DRAM-based main memory because of its higher energy efficiency and similar latency to DRAM. However, simply replacing DRAM with STT-MRAM without optimizations severely limits STT-MRAM from exploiting its full potential. STT-MRAM employs costly sense amplifiers that demand an order of magnitude more area and power than DRAM. To manage the high cost, STT-MRAM shares one sense amplifier across multiple bit-lines, exploiting the non-destructive nature of its read operation. This sense amplifier sharing reduces the size of row buffers; as a result, it incurs higher activation energy and lower performance. Other issues arise if STT-MRAM is required to be compatible with DRAM interfaces and policies. To address these challenges in a cost-effective manner, we propose SMART that, unlike DRAM and conventional STT-MRAM, waits to do bit-line sensing until after receiving a column access command instead of a row activation command. This results in several benefits: larger pages, fewer sense amplifiers, lower activation power, higher bank-level parallelism, shorter latency, fewer address pins, and more efficient repairing of defective columns than conventional STT-MRAM. Our evaluation shows that SMART consumes lower energy while providing higher performance than conventional STT-MRAM and DRAM. Additionally, SMART consumes less area compared to conventional STT-MRAM.


  • A Study of Memory Placement on Hardware-Assisted Tiered Memory Systems

    by Wonkyo Choe, Jonghyeon Kim, Jeongseob Ahn

    Recent advances in memory technology, memory hierarchy is becoming diverse with performance-differentiated memory such as high bandwidth memory (HBM) and non-volatile memory (NVM) in modern computer systems. However, the current memory placement has been designed with the assumption that all the memory has the same capabilities based on DRAM. In this paper, we analyze memory placement schemes in state-of-the-art Linux systems to the hw-assisted tiered memory system. Our analysis is conducted on the real system equipped with Intel's Optane Persistent Memory, enabling tiered memory with DRAM (fast) and Optane (Slow) memory. We observe that tiered memory systems do not exhibit that accessing the local memory provides better performance than that of the remote memory because local Optane memory is slower than remote DRAM. Due to this distinct characteristic, there are several inefficiencies in the commodity operating systems. To make use of tiered memory systems efficiently, this paper explores the design space of practical software solutions, which can be currently applicable in the Linux system.


  • Reconciling Time Slice Conflicts of Virtual Machines With Dual Time Slice for Clouds

    by Taeklim Kim, Chang Hyun Park, Jaehyuk Huh, Jeongseob Ahn

    The proliferation of system virtualization poses a new challenge for the coarse-grained time sharing techniques for consolidation, since operating systems are running on virtual CPUs. The current system stack was designed under the assumption that operating systems can seize CPU resources at any moment. However, for the guest operating system on a virtual machine (VM), such assumption cannot be guaranteed, since virtual CPUs of VMs share a limited number of physical cores. Due to the time-sharing of physical cores, the execution of a virtual CPU is not contiguous, with a gap between the virtual and real time spaces. Such a virtual time.


  • LightCert: On designing a lighter certificate for resource‐limited Internet‐of‐Things devices

    by Hyuksang Kwon, Jeongseob Ahn, JeongGil Ko

    Internet‐of‐Things (IoT) devices are typically resource constrained in terms of computing capabilities and battery power. Despite the efforts from the Internet Engineering Task Force (IETF) to established standards for IoT such as IPv6 over low‐power wireless personal area networks (6LoWPAN), routing protocol for low‐power lossy networks (RPL), and constrained application protocol (CoAP), certificate‐based Internet security protocols have not been fully addressed yet.We see the main cause of this being the size of the X.509‐based Internet certificates. Typically being 1 to 2 kB, the large size of these certificates can cause IEEE 802.15.4‐based IoT nodes to fragment the certificate into many smaller packet‐size chunks, which causes many packet transmissions to occur in the network. This work presents LightCert, a lightweight scheme to compress the size of the security certificates using the similarity of contents in X.509 certificates. Specifically, LightCert identifies common fields in a certificate and suppresses the transmission of these contents within the IoT subnet scope. This allows LightCert nodes to minimize the packet transmission overhead for supporting certificate‐based security mechanisms such as datagram transport layer security (DTLS), by as much as ∼37%. The added overhead of exchanging certificates when using LightCert is kept low to as much as ∼5 mJ of energy and ∼0.48 seconds of latency.