Yichao Yuan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yichao Yuan has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Nishil Talati
University of Illinois Urbana-Champaign, Urbana, Illinois, USA
3 shared publications - Divyam Sharma
University of Michigan, Ann Arbor, MI, USA
1 shared publication - Lin Ma
Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, Michigan, USA
1 shared publication - Mosharaf Chowdhury
University of Michigan, Ann Arbor, Michigan, USA
1 shared publication - Souvik Kundu
Intel Labs
1 shared publication - Yuchen Xia
University of Michigan, Ann Arbor, MI, USA
1 shared publication
Latest Publications
- Open Access
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 573-593
KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
- 10 April 2027
- Published by Association for Computing Machinery (ACM)
- p. 573-593
Open AccessKAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
- Open Access
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 415-429
MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
Mixture of Experts (MoE) LLMs, with sparse activation patterns, offer a promising approach to scaling language models while avoiding proportional cost increases. However, their large parameter sizes pose deployment challenges in resource-constrained environments with limited GPU memory. Typical deployments use CPU-GPU hybrid execution, where the GPU handles compute-intensive GEMM operations and the CPU processes the attention mechanism. This setup introduces a challenge: optimizing resource utilization across CPU and GPU. Prior work designs system optimizations based on performance models with a limited scope that don’t capture complex hardware-system interactions. Therefore, they neither identify nor achieve hardware limits. This paper presents MoE-Lens, a high-throughput MoE LLM inference system designed for resource-constrained environments. MoE-Lens utilizes a novel performance model that holistically analyzes CPU memory capacity, GPU compute power, and workload characteristics to understand the theoretical performance upper bound of MoE inference. It also captures system execution mechanisms, including workload scheduling and the effects of paged KV cache, to identify key hardware bottlenecks and predict achievable throughput. Guided by such a performance model, MoE-Lens introduces a resource-aware scheduler, an execution engine that overlaps computation, a data transfer mechanism for model weights, and an optimized CPU-based attention implementation. Evaluated on diverse MoE models and datasets, MoE-Lens outperforms the state-of-the-art MoE-Lightening by 4.6 × (up to 25.5 ×), with our theoretical model predicting performance with an average 94% accuracy.
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 415-429
Open AccessMoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
Mixture of Experts (MoE) LLMs, with sparse activation patterns, offer a promising approach to scaling language models while avoiding proportional cost increases. However, their large parameter sizes pose deployment challenges in resource-constrained environments with limited GPU memory. Typical deployments use CPU-GPU hybrid execution, where the GPU handles compute-intensive GEMM operations and the CPU processes the attention mechanism. This setup introduces a challenge: optimizing resource utilization across CPU and GPU. Prior work designs system optimizations based on performance models with a limited scope that don’t capture complex hardware-system interactions. Therefore, they neither identify nor achieve hardware limits. This paper presents MoE-Lens, a high-throughput MoE LLM inference system designed for resource-constrained environments. MoE-Lens utilizes a novel performance model that holistically analyzes CPU memory capacity, GPU compute power, and workload characteristics to understand the theoretical performance upper bound of MoE inference. It also captures system execution mechanisms, including workload scheduling and the effects of paged KV cache, to identify key hardware bottlenecks and predict achievable throughput. Guided by such a performance model, MoE-Lens introduces a resource-aware scheduler, an execution engine that overlaps computation, a data transfer mechanism for model weights, and an optimized CPU-based attention implementation. Evaluated on diverse MoE models and datasets, MoE-Lens outperforms the state-of-the-art MoE-Lightening by 4.6 × (up to 25.5 ×), with our theoretical model predicting performance with an average 94% accuracy.
- Open Access
- 11 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 163-182
MoDM: Efficient Serving for Image Generation via M ixture- o f- D iffusion M odels
- 11 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 163-182
Open AccessMoDM: Efficient Serving for Image Generation via M ixture- o f- D iffusion M odels