ZWZicong Wang

Zicong Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Zicong Wang has a total of 28 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    AdaptiveKV: Accelerating KV Cache Offloading with a Bandwidth-Adaptive Memory Allocation Mechanism

    by Yibo Tang, Lizhou Wu, Yang Ou, Sunfeng Gao, Yanjing Wang, Zicong Wang, Xingyun Qi, Jiaqing Xu, Fangxu Lv, Liquan Xiao, et al.

    The explosive growth of key-value (KV) cache size in large language model (LLM) inference poses a key challenge to the limited HBM of GPU. Offloading KV cache to host memory has become a prevalent mitigation method. However, the limited host DDR bandwidth, especially in multi-GPU inference scenarios, often leads to offloading bottlenecks, thereby restricting inference speed. Compute express link (CXL) offers a promising alternative to expand host memory capacity and bandwidth on demand. In this article, we present a bandwidth-oriented memory allocation mechanism, named AdaptiveKV, which is self-adaptive to CXL-enabled memory pools and KV cache offloading scales for LLM inference acceleration. Our systematic profiling of CXL-HBM memory bandwidth under GPU workloads reveals that conventional memory strategies neglect dynamic memory bandwidth fluctuations and various CXL memory characteristics, leading to suboptimal memory utilization. Motivated by these insights, AdaptiveKV implements three core designs: (1) a GPU memory conch model to guide memory allocation strategies, (2) a runtime predictor to predict optimal memory allocation ratios, and (3) a dynamic interleaving strategy to allocate memory pages across available NUMA nodes. Experimental results suggest that AdaptiveKV achieves a maximum speedup of 1.90× in LLM inference throughput compared with the state-of-the-art strategies. To further explore AdaptiveKV’s applicability boundary, we also present an FPGA-based CXL memory emulator with configurable performance, revealing that a CXL-to-DDR bandwidth ratio exceeding 8% yields at least a 5% speedup in LLM inference.

  • Open Access

    Elevating Temporal Prefetching Through Instruction Correlation

    by Shuiyi He, Zicong Wang, Xuan Tang, Hao Tang, Dezun Dong, Liquan Xiao


  • Open Access

    vtism: Efficient Tiered Memory Management for Virtual Machines with CXL

    by Zhixing Lu, Yang Ou, Lizhou Wu, Zicong Wang, Mingche Lai, Xuran Ge, Zhenlong Song, Min Xie


  • CXL-DMSim: A Full-System CXL Disaggregated Memory Simulator With Comprehensive Silicon Validation

    by Yanjing Wang, Lizhou Wu, Wentao Hong, Yang Ou, Zicong Wang, Sunfeng Gao, Jie Zhang, Sheng Ma, Dezun Dong, Xingyun Qi, et al.

    Compute Express Link (CXL) has emerged as a key enabler of memory disaggregation for future heterogeneous computing systems to expand memory on demand and improve resource utilization. However, CXL is still in its infancy stage and lacks commodity products on the market, thus necessitating a reliable system-level simulation tool for research and development. In this article, we propose CXL-DMSim (open-sourced at https://github.com/ferry-hhh/CXL-DMSim), an open-source full-system (FS) simulator to simulate CXL disaggregated memory systems with high fidelity at a gem5-comparable simulation speed. CXL-DMSim incorporates a flexible CXL memory expander model along with its associated device driver, and CXL protocol support with CXL.io and CXL.mem. It can operate in both the app-managed (AM) mode and the kernel-managed (KM) mode, with the latter using a dedicated NUMA-compatible mechanism. The simulator has been rigorously verified against a real hardware testbed with both FPGA- and ASIC-based CXL memory devices, which demonstrates the qualification of CXL-DMSim in simulating the characteristics of various CXL memory devices at an average simulation error of 3.4%. The experimental results using LMbench and STREAM benchmarks suggest that the CXL-FPGA memory exhibits a $\sim 2.88\times $ higher latency than local DDR, while the CXL-ASIC latency is $\sim 2.18\times $ ; CXL-FPGA achieves 45%–69% of local DDR memory bandwidth, whereas the number for CXL-ASIC is 82%–83%. The study also reveals that CXL memory can significantly enhance the performance of memory-intensive applications, improving by $23\times $ at most with limited local memory for Viper key–value database and approximately 60% in memory-bandwidth-sensitive scenarios such as MERCI. Moreover, the simulator’s observability and expandability are showcased with detailed case studies, highlighting its great potential for research on future CXL-interconnected hybrid memory pools.


  • Open Access

    Chimera: Leveraging Hybrid Offsets for Efficient Data Prefetching

    by Shuiyi He, Zicong Wang, Xuan Tang, Qiyao Sun, Dezun Dong

    Data prefetching is an essential technique in contemporary high-performance processors for mitigating the effects of long-latency memory accesses. With the increasing demand for prefetcher to learn complex memory access patterns, many state-of-the-art prefetchers adopt methods such as using the program counter or access delta to separate memory access streams. This allows them to learn detailed memory access features and thereby improve memory system performance. However, the separation-based approach is prone to missing global correlations, leading to miss prefetching opportunities. In this paper, we propose Chimera, a hybrid offsets prefetcher that captures multiple offset features from the overall stream of memory access instructions, thus overcoming the drawbacks of traditional prefetchers that tend to lose memory access information when learning from one-sided features. We evaluated Chimera using SPEC CPU 2006 and 2017 through simulation, and the results show that Chimera improves system performance by 39.5% over a baseline with no data prefetcher and by 6.6% over the state-of-the-art data prefetcher.


  • Open Access

    Cache Access Fairness in 3D Mesh-Based NUCA

    by Zicong Wang, Xiaowen Chen, Zhonghai Lu, Yang Guo

    Given the increase in cache capacity over the past few decades, cache access efficiency has come to play a critical role in determining system performance. To ensure efficient utilization of the cache resources, non-uniform cache architecture (NUCA) has been proposed to allow for a large capacity and a short access latency. With the support of networks-on-chip (NoC), NUCA is often employed to organize the last level cache. However, this method also hurts cache access fairness, which denotes the degree of non-uniformity for cache access latencies. This drop in fairness can result in an increased number of cache accesses with overhigh latency, which leads to a bottleneck in system performance. This paper investigates the cache access fairness in the context of NoC-based 3-D chip architecture, and provides new insights into 3-D architecture design. We propose fair-NUCA (F-NUCA), a co-design scheme intended to optimize cache access fairness. In F-NUCA, we strive to improve fairness by equalizing cache access latencies. To achieve this goal, the memory mapping and the channel width are both redistributed non-uniformly, thereby equalizing the non-contention and contention latencies, respectively. The experimental results reveal that F-NUCA can effectively improve cache access fairness. When F-NUCA is compared with the traditional static NUCA in a simulation with PARSEC benchmarks, the average reductions in average latency and latency standard deviation are 4.64%/9.38% for a $4 \times 4 \times 2$ mesh network, as well as 6.31%/13.51% for a $4 \times 4 \times 4$ mesh network. In addition, a 4.0%/6.4% improvement in system throughput can be achieved for the two scales of mesh networks, respectively.