Yifan Qiao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yifan Qiao has a total of 57 co-authors. Here are the top co-authors by number of shared publications:
- Jiarong Xing
Rice University, Houston, Texas, USA
5 shared publications - Guoqing Harry Xu
University of California, Los Angeles, Los Angeles, USA
4 shared publications - Ion Stoica
University of California, Berkeley, Berkeley, USA
4 shared publications - Bei Ouyang
Rice University, Houston, Texas, USA
3 shared publications - Chenxi Wang
University of Chinese Academy of Sciences, Beijing, China
3 shared publications - Junyi Shu
University of California, Los Angeles, Los Angeles, USA
2 shared publications
Latest Publications
- Open Access
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 136-152
Breaking the Tradeoff: Elastic and Isolated GPU Sharing with GVM
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 136-152
Open AccessBreaking the Tradeoff: Elastic and Isolated GPU Sharing with GVM
- Open Access
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 380-398
Firework: Efficient and Transparent DSM with Hardware-Coherent CXL Shared Memory
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 380-398
Open AccessFirework: Efficient and Transparent DSM with Hardware-Coherent CXL Shared Memory
- Open Access
- 20 July 2026
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 31-33
UniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
Prefix caching is a key performance optimization in Large Language Model (LLM) serving systems, enabling reuse of attention Key-Value (KV) states across requests with shared prompt prefixes. However, the size of GPU memory limits cache capacity, making the eviction policy a critical factor in overall system performance. Existing systems primarily rely on simple heuristics, such as LRU, and apply the same policy across task categories, implicitly assuming homogeneous workloads. In practice, however, modern LLMs serve heterogeneous workloads that mix multi-turn conversational traffic with diverse single-turn API requests, leading to fundamentally different prefix reuse patterns. In this work, we first design a trace-driven prefix cache simulator built on vLLM to systematically characterize prefix reuse across representative workloads. Our analysis reveals two dominant reuse patterns---session reuse and structural reuse---that vary significantly across task types. Motivated by these observations, we propose \sys, a unified eviction policy that jointly captures both reuse patterns and dynamically balances cache allocation across tasks. When implemented in vLLM, UniCache achieves substantial improvements under heterogeneous workloads, improving prefix cache hit rates by up to 17.32% and reducing inference latency by up to 3.63× compared to existing policies.
- 20 July 2026
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 31-33
Open AccessUniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
Prefix caching is a key performance optimization in Large Language Model (LLM) serving systems, enabling reuse of attention Key-Value (KV) states across requests with shared prompt prefixes. However, the size of GPU memory limits cache capacity, making the eviction policy a critical factor in overall system performance. Existing systems primarily rely on simple heuristics, such as LRU, and apply the same policy across task categories, implicitly assuming homogeneous workloads. In practice, however, modern LLMs serve heterogeneous workloads that mix multi-turn conversational traffic with diverse single-turn API requests, leading to fundamentally different prefix reuse patterns. In this work, we first design a trace-driven prefix cache simulator built on vLLM to systematically characterize prefix reuse across representative workloads. Our analysis reveals two dominant reuse patterns---session reuse and structural reuse---that vary significantly across task types. Motivated by these observations, we propose \sys, a unified eviction policy that jointly captures both reuse patterns and dynamically balances cache allocation across tasks. When implemented in vLLM, UniCache achieves substantial improvements under heterogeneous workloads, improving prefix cache hit rates by up to 17.32% and reducing inference latency by up to 3.63× compared to existing policies.
- Open Access
- 8 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 31-33
UniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
- 8 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 31-33
Open AccessUniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
- Open Access
- 29 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Vol. 10 (2) , 1-27
UniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
Prefix caching is a key performance optimization in Large Language Model (LLM) serving systems, enabling reuse of attention Key-Value (KV) states across requests with shared prompt prefixes. However, the size of GPU memory limits cache capacity, making the eviction policy a critical factor in overall system performance. Existing systems primarily rely on simple heuristics, such as LRU, and apply the same policy across task categories, implicitly assuming homogeneous workloads. In practice, however, modern LLMs serve heterogeneous workloads that mix multi-turn conversational traffic with diverse single-turn API requests, leading to fundamentally different prefix reuse patterns. In this work, we first design a trace-driven prefix cache simulator built on vLLM to systematically characterize prefix reuse across representative workloads. Our analysis reveals two dominant reuse patterns---session reuse and structural reuse---that vary significantly across task types. Motivated by these observations, we propose UniCache, a unified eviction policy that jointly captures both reuse patterns and dynamically balances cache allocation across tasks. When implemented in vLLM, UniCache achieves substantial improvements under heterogeneous workloads, improving prefix cache hit rates by up to 17.32% and reducing inference latency by up to 3.63× compared to existing policies.
- 29 May 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Vol. 10 (2) , 1-27
Open AccessUniCache: Unifying Prefix Cache Eviction for Heterogeneous LLM Serving Workloads
Prefix caching is a key performance optimization in Large Language Model (LLM) serving systems, enabling reuse of attention Key-Value (KV) states across requests with shared prompt prefixes. However, the size of GPU memory limits cache capacity, making the eviction policy a critical factor in overall system performance. Existing systems primarily rely on simple heuristics, such as LRU, and apply the same policy across task categories, implicitly assuming homogeneous workloads. In practice, however, modern LLMs serve heterogeneous workloads that mix multi-turn conversational traffic with diverse single-turn API requests, leading to fundamentally different prefix reuse patterns. In this work, we first design a trace-driven prefix cache simulator built on vLLM to systematically characterize prefix reuse across representative workloads. Our analysis reveals two dominant reuse patterns---session reuse and structural reuse---that vary significantly across task types. Motivated by these observations, we propose UniCache, a unified eviction policy that jointly captures both reuse patterns and dynamically balances cache allocation across tasks. When implemented in vLLM, UniCache achieves substantial improvements under heterogeneous workloads, improving prefix cache hit rates by up to 17.32% and reducing inference latency by up to 3.63× compared to existing policies.
- Open Access
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 255-273
BlendServe: Optimizing Offline Inference with Resource-Aware Batching
- 22 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 255-273
Open AccessBlendServe: Optimizing Offline Inference with Resource-Aware Batching
- Open Access
- 17 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 131-138
Lost in Translation: The Search for Meaning in Network-Attached AI Accelerator Disaggregation
- 17 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 131-138
Open AccessLost in Translation: The Search for Meaning in Network-Attached AI Accelerator Disaggregation
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 286-304
Orthrus: Efficient and Timely Detection of Silent User Data Corruption in the Cloud with Resource-Adaptive Computation Validation
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 286-304
Open AccessOrthrus: Efficient and Timely Detection of Silent User Data Corruption in the Cloud with Resource-Adaptive Computation Validation
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 399-414
PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 399-414
Open AccessPrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications
- Open Access
- 25 August 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 43 (3) , 1-32
Lining up Garbage Collection and Application for a Far-Memory-Friendly Runtime
Far-memory techniques that enable applications to use remote memory are increasingly appealing in modern data centers, supporting applications’ large memory footprint and improving machines’ resource utilization. Unfortunately, most far-memory techniques focus on OS-level optimizations and are agnostic to managed runtimes and garbage collections (GC) underneath applications written in high-level languages. With different object-access patterns from applications, GC can severely interfere with existing far-memory techniques, breaking remote memory prefetching algorithms and causing severe local-memory misses. We developed MemLiner, a runtime technique that improves the performance of far-memory systems by aligning memory accesses from application and GC threads so that they follow similar memory access paths, thereby (1) reducing the local-memory working set and (2) improving remote-memory prefetching through simplified memory access patterns. We implemented MemLiner in two widely used GCs in OpenJDK: G1 and Shenandoah. Our evaluation with a range of widely deployed cloud systems shows that MemLiner improves applications’ end-to-end performance by up to 3.3× and reduces applications’ tail latency by up to 220.0× .
- 25 August 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer Systems
- Vol. 43 (3) , 1-32
Open AccessLining up Garbage Collection and Application for a Far-Memory-Friendly Runtime
Far-memory techniques that enable applications to use remote memory are increasingly appealing in modern data centers, supporting applications’ large memory footprint and improving machines’ resource utilization. Unfortunately, most far-memory techniques focus on OS-level optimizations and are agnostic to managed runtimes and garbage collections (GC) underneath applications written in high-level languages. With different object-access patterns from applications, GC can severely interfere with existing far-memory techniques, breaking remote memory prefetching algorithms and causing severe local-memory misses. We developed MemLiner, a runtime technique that improves the performance of far-memory systems by aligning memory accesses from application and GC threads so that they follow similar memory access paths, thereby (1) reducing the local-memory working set and (2) improving remote-memory prefetching through simplified memory access patterns. We implemented MemLiner in two widely used GCs in OpenJDK: G1 and Shenandoah. Our evaluation with a range of widely deployed cloud systems shows that MemLiner improves applications’ end-to-end performance by up to 3.3× and reduces applications’ tail latency by up to 220.0× .