Yang Kong
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yang Kong has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Tao Huang
Alibaba Cloud Computing, Hangzhou, Zhejiang, China
3 shared publications - Xinjun Yang
Alibaba Cloud Computing, Sunnyvale, USA
3 shared publications - Feifei Li
Alibaba Cloud, Hangzhou, China
2 shared publications - Guoqiang Xu
Alibaba Cloud Computing, Hangzhou, China
2 shared publications - Hao Chen
Alibaba Cloud Computing, Hangzhou, China
2 shared publications - Wenpu Hu
Alibaba Cloud Computing, Hangzhou, Zhejiang, China
2 shared publications
Latest Publications
- Open Access
- 2 April 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (1) , 1-29
Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
The rapid increase in LLM model sizes and the growing demand for long-context inference have made memory a critical bottleneck in GPU-accelerated LLM serving. Although high-bandwidth memory (HBM) on GPUs offers fast access, its limited capacity necessitates reliance on host memory (CPU DRAM) to support large KVCache. However, the maximum DRAM capacity is constrained by the limited number of memory channels per CPU socket. To overcome this limitation, current systems often adopt RDMA-based disaggregated memory pools, which introduce significant challenges including high access latency, complex communication protocols, and synchronization overhead. Fortunately, the emerging CXL technology introduces new opportunities in KVCache design. In this paper, we propose Beluga , a novel memory architecture that enables GPUs and CPUs to access a shared, large-scale memory pool through CXL switches. By supporting native load/store access semantics over the CXL fabric, our design delivers near-local memory latency, while reducing programming complexity and minimizing synchronization overhead. We conduct a systematic characterization of CXL-based memory pool and propose a set of design guidelines. Based on Beluga , we design and implement Beluga -KVCache, a system tailored for managing the large-scale KVCache for LLM inference. Beluga -KVCache achieves an 89.6% reduction in Time-To-First-Token (TTFT) and 7.35x throughput improvement in vLLM compared to RDMA-based solutions. To the best of our knowledge, Beluga is the first system that enables GPUs to directly access large-scale memory pools through CXL switches (Marvell XConn XC50256), marking a significant step toward low-latency, shared access to vast memory resources by GPUs.
- 2 April 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 4 (1) , 1-29
Open AccessBeluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
The rapid increase in LLM model sizes and the growing demand for long-context inference have made memory a critical bottleneck in GPU-accelerated LLM serving. Although high-bandwidth memory (HBM) on GPUs offers fast access, its limited capacity necessitates reliance on host memory (CPU DRAM) to support large KVCache. However, the maximum DRAM capacity is constrained by the limited number of memory channels per CPU socket. To overcome this limitation, current systems often adopt RDMA-based disaggregated memory pools, which introduce significant challenges including high access latency, complex communication protocols, and synchronization overhead. Fortunately, the emerging CXL technology introduces new opportunities in KVCache design. In this paper, we propose Beluga , a novel memory architecture that enables GPUs and CPUs to access a shared, large-scale memory pool through CXL switches. By supporting native load/store access semantics over the CXL fabric, our design delivers near-local memory latency, while reducing programming complexity and minimizing synchronization overhead. We conduct a systematic characterization of CXL-based memory pool and propose a set of design guidelines. Based on Beluga , we design and implement Beluga -KVCache, a system tailored for managing the large-scale KVCache for LLM inference. Beluga -KVCache achieves an 89.6% reduction in Time-To-First-Token (TTFT) and 7.35x throughput improvement in vLLM compared to RDMA-based solutions. To the best of our knowledge, Beluga is the first system that enables GPUs to directly access large-scale memory pools through CXL switches (Marvell XConn XC50256), marking a significant step toward low-latency, shared access to vast memory resources by GPUs.
- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (4) , 1391-1402
CXLock: Efficient and Scalable Lock Management for CXL-Enabled Distributed Systems
Efficient lock management is critical in distributed systems with shared resources, especially in the context of enhanced computational power and reduced processing time. While numerous studies have leveraged Remote Direct Memory Access (RDMA) for distributed locking, these network-based approaches suffer from high network latency and jitter. The emergence of the Compute Express Link (CXL) protocol provides high-speed, low-latency connections between host processors and memory devices, presenting a compelling alternative for the lock system design. This paper introduces CXLock, the industry’s first distributed lock management system based on CXL 2.0 switches. CXLock decouples lock management into client-side queuing and manager-side arbitration, resolving race conditions of multiple clients without using remote atomics. To handle the lack of hardware-enforced coherence in CXL 2.0, CXLock employs a software-based coherence model that enables memory sharing across multiple hosts with cacheable and write-back attributes. CXLock is implemented and evaluated on a real rack-scale hardware platform containing a CXL switch and multiple host nodes. The results show that CXLock delivers an average throughput improvement of 3.7$\boldsymbol{\times}$ over the RDMA-based solutions. Moreover, CXLock reduces the lock grant latency by up to 76% in both contention-free and contended scenarios.- 28 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 75 (4) , 1391-1402
CXLock: Efficient and Scalable Lock Management for CXL-Enabled Distributed Systems
Efficient lock management is critical in distributed systems with shared resources, especially in the context of enhanced computational power and reduced processing time. While numerous studies have leveraged Remote Direct Memory Access (RDMA) for distributed locking, these network-based approaches suffer from high network latency and jitter. The emergence of the Compute Express Link (CXL) protocol provides high-speed, low-latency connections between host processors and memory devices, presenting a compelling alternative for the lock system design. This paper introduces CXLock, the industry’s first distributed lock management system based on CXL 2.0 switches. CXLock decouples lock management into client-side queuing and manager-side arbitration, resolving race conditions of multiple clients without using remote atomics. To handle the lack of hardware-enforced coherence in CXL 2.0, CXLock employs a software-based coherence model that enables memory sharing across multiple hosts with cacheable and write-back attributes. CXLock is implemented and evaluated on a real rack-scale hardware platform containing a CXL switch and multiple host nodes. The results show that CXLock delivers an average throughput improvement of 3.7$\boldsymbol{\times}$ over the RDMA-based solutions. Moreover, CXLock reduces the lock grant latency by up to 76% in both contention-free and contended scenarios.- Open Access
- 22 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 689-702
Unlocking the Potential of CXL for Disaggregated Memory in Cloud-Native Databases
- 22 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 689-702
Open AccessUnlocking the Potential of CXL for Disaggregated Memory in Cloud-Native Databases
- 27 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (3) , 902-914
Gem5Tune: A Parameter Auto-Tuning Framework for Gem5 Simulator to Reduce Errors
Computer architecture simulators are widely used to explore new architectures, e.g., the gem5 simulator. However, gem5 has significant performance errors that may lead to misleading research results. Researchers typically reduce errors with the target machine by manual calibration methods, which are time-consuming and require significant expertise. This paper presents gem5Tune, a parameter auto-tuning framework for the gem5 simulator to reduce errors. Applying black-box optimization (BBO) methods, recommended for TPE-based Bayesian optimization, gem5Tune minimizes the error between gem5 and the target machine within a limited number of iterations. Three optimization methods, instruction calibration, sensitivity analysis, and dynamic pruning, are proposed to accelerate the error convergence. Experimental results show that compared to the manual calibration method, gem5Tune significantly reduces performance errors between gem5 and three modern ARM servers by more than 10% (13.83%, 10.86%, and 25.22%, respectively) for SPEC CPU benchmarks. It also scales effectively to PARSEC and SPLASH-2x benchmarks and reduces the errors of architectural events.
- 27 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (3) , 902-914
Gem5Tune: A Parameter Auto-Tuning Framework for Gem5 Simulator to Reduce Errors
Computer architecture simulators are widely used to explore new architectures, e.g., the gem5 simulator. However, gem5 has significant performance errors that may lead to misleading research results. Researchers typically reduce errors with the target machine by manual calibration methods, which are time-consuming and require significant expertise. This paper presents gem5Tune, a parameter auto-tuning framework for the gem5 simulator to reduce errors. Applying black-box optimization (BBO) methods, recommended for TPE-based Bayesian optimization, gem5Tune minimizes the error between gem5 and the target machine within a limited number of iterations. Three optimization methods, instruction calibration, sensitivity analysis, and dynamic pruning, are proposed to accelerate the error convergence. Experimental results show that compared to the manual calibration method, gem5Tune significantly reduces performance errors between gem5 and three modern ARM servers by more than 10% (13.83%, 10.86%, and 25.22%, respectively) for SPEC CPU benchmarks. It also scales effectively to PARSEC and SPLASH-2x benchmarks and reduces the errors of architectural events.