DLDongyang Li

Dongyang Li

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Dongyang Li has a total of 29 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Faster-MoA: Low-Latency Tree-Structured MoA Serving with Early Exit and Agent-Aware Prefill-Decode Overlap

    by Zijun Wang, Yijiahao Qi, Hanqiu Chen, Zishen Wan, Gongjin Sun, Dongyang Li, Shuyi Pei, Cong Hao

  • Open Access

    LLM KV Cache Storage Using CXL Memory

    by Kwan Huen, Gongjin Sun, Mehdi Nik, Caroline Kahn, Elham Movahedian, Senthil Murugesapandian, Michael Wasef, Shuyu Lyu, Dongyang Li, Edmund Au, et al.

  • Open Access

    CMM-H PMEM: Performance Characterization of CXL Large Capacity Persistent Memory

    by Gongjin Sun, Mehdi Nik, Senthil Murugesapandian, Dongyang Li, Kwan Huen, Xuebin Yao, Caroline Kahn, Amir Beygi, Vitorio Cargnini, Ramdas Kachare

    System architectures place memory in close proximity of CPU to achieve high performance. However, this directly and locally attached memory comes with constraints. Compute Express Link (CXL) enables robust memory disaggregation and thereby opens up new possibilities to address some of the direct-attached memory constraints. Samsung CMM-H is a CXL memory module that combines best aspects of DRAM and NAND in a novel architecture to provide a large capacity persistent memory (PMEM) solution. While NVMe SSDs offer capacity, they suffer from higher latency and inevitable write amplification when subjected to the fine-grained writes of data-intensive workloads. This paper presents a comprehensive characterization of the CMM-H PMEM solution. CMM-H device performance profile is compared to NVMe SSD. It achieves up to 100M random read IOPS, and latencies an order of magnitude lower for small I/Os. The large built-in DRAM cache is the architectural mechanism that enables this performance. The same mechanism also achieves a significant reduction in data written to the backend NAND thereby reducing write amplification factor (WAF). Lower WAF in turn provides benefits such as higher media endurance, lower power consumption, and lower latency. Direct measurement of real-world case studies demonstrates this mechanism reduces WAF by over 200x. This unique combination of high performance and significant write reduction establishes CMM-H as a compelling device solution in the PMEM landscape.


  • Open Access

    ICGMM: CXL-enabled Memory Expansion with Intelligent Caching Using Gaussian Mixture Model

    by Hanqiu Chen, Yitu Wang, Vitorio Cargnini, Mohammadreza Soltaniyeh, Dongyang Li, Gongjin Sun, Pradeep Subedi, Andrew Chang, Yiran Chen, Cong Hao

    Compute Express Link (CXL) emerges as a solution for wide gap between computational speed and data communication rates among host and multiple devices. It fosters a unified and coherent memory space between host and CXL storage devices such as such as Solid-state drive (SSD) for memory expansion, with a corresponding DRAM implemented as the device cache. However, this introduces challenges such as substantial cache miss penalties, sub-optimal caching due to data access granularity mismatch between the DRAM "cache" and SSD "memory", and inefficient hardware cache management. To address these issues, we propose a novel solution, named ICGMM, which optimizes caching and eviction directly on hardware, employing a Gaussian Mixture Model (GMM)-based approach. We prototype our solution on an FPGA board, which demonstrates a noteworthy improvement compared to the classic Least Recently Used (LRU) cache strategy. We observe a decrease in the cache miss rate ranging from 0.32% to 6.14%, leading to a substantial 16.23% to 39.14% reduction in the average SSD access latency. Furthermore, when compared to the state-of-the-art Long Short-Term Memory (LSTM)-based cache policies, our GMM algorithm on FPGA showcases an impressive latency reduction of over 10,000 times. Remarkably, this is achieved while demanding much fewer hardware resources.