Chengzhi Lu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Chengzhi Lu has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Cheng-Zhong Xu
State Key Laboratory of IoTSC, University of Macau, Macau, Macao
6 shared publications - Kejiang Ye
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
6 shared publications - Huanle Xu
University of Macau
3 shared publications - Shijie Peng
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
3 shared publications - Yanying Lin
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China
3 shared publications - Shuaipeng Wu
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
2 shared publications
Latest Publications
- Open Access
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 348-358
ReliefServe: Relieving GPU Pressure in Multi-Model Serving via Selective CPU Escape
- 27 September 2026
- Published by Association for Computing Machinery (ACM)
- p. 348-358
Open AccessReliefServe: Relieving GPU Pressure in Multi-Model Serving via Selective CPU Escape
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 56-71
High Throughput and Low Latency LLM Serving via Adaptive KV Caching
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 56-71
Open AccessHigh Throughput and Low Latency LLM Serving via Adaptive KV Caching
- Open Access
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 72-88
FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
- 26 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 72-88
Open AccessFlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters
- 28 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 18 (5) , 3164-3176
Serving LLM in Distributed GPU Cluster With Fine-Grain Pipeline Constraints
As Large Language Models (LLMs) continue to advance, their parameter sizes are growing exponentially-far outpacing hardware capabilities. This widening gap necessitates distributed computing through pipeline parallelism for efficient inference. However, the uneven distribution of requests across pipeline stages creates significant performance bottlenecks in real-world deployments. To address this challenge, we present Planck, a performance optimization framework specifically designed for distributed LLM inference. Planck implements fine-grained control through two key mechanisms: a progressive SLO allocation strategy that dynamically adjusts time constraints based on workload patterns, and stage-specific performance controllers that prevent bottlenecks before they cascade through the system. By intelligently balancing resources across pipeline stages, Planck effectively eliminates queue buildup-essentially preventing traffic congestion before it forms. Evaluation using diverse workloads in real cloud environments demonstrates that Planck reduces P99 tail latency by up to 18% and decreases the longest queue lengths by as much as 47.8% across pipeline stages, significantly improving both system responsiveness and resource utilization.
- 28 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Services Computing
- Vol. 18 (5) , 3164-3176
Serving LLM in Distributed GPU Cluster With Fine-Grain Pipeline Constraints
As Large Language Models (LLMs) continue to advance, their parameter sizes are growing exponentially-far outpacing hardware capabilities. This widening gap necessitates distributed computing through pipeline parallelism for efficient inference. However, the uneven distribution of requests across pipeline stages creates significant performance bottlenecks in real-world deployments. To address this challenge, we present Planck, a performance optimization framework specifically designed for distributed LLM inference. Planck implements fine-grained control through two key mechanisms: a progressive SLO allocation strategy that dynamically adjusts time constraints based on workload patterns, and stage-specific performance controllers that prevent bottlenecks before they cascade through the system. By intelligently balancing resources across pipeline stages, Planck effectively eliminates queue buildup-essentially preventing traffic congestion before it forms. Evaluation using diverse workloads in real cloud environments demonstrates that Planck reduces P99 tail latency by up to 18% and decreases the longest queue lengths by as much as 47.8% across pipeline stages, significantly improving both system responsiveness and resource utilization.
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 589-604
Multiplexing Dynamic Deep Learning Workloads with SLO-awareness in GPU Clusters
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 589-604
Open AccessMultiplexing Dynamic Deep Learning Workloads with SLO-awareness in GPU Clusters
- Open Access
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 416-432
Understanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms
To fully utilize computing resources, cloud providers such as Google and Alibaba choose to co-locate online services with batch processing applications in their data centers. By implementing unified resource management policies, different types of complex computing jobs request resources in a consistent way, which can help data centers achieve global optimal scheduling and provide computing power with higher quality. To understand this new scheduling paradigm, in this paper, we first present an in-depth study of Alibaba's unified scheduling workloads. Our study focuses on the characterization of resource utilization, the application running performance, and scheduling scalability. We observe that although computing resources are significantly over-committed under unified scheduling, the resource utilization in Alibaba data centers is still low. In addition, existing resource usage predictors tend to make severe overestimations. At the same time, tasks within the same application behave fairly consistently, and the running performance of tasks can be well-profiled with respect to resource contention on the corresponding physical host. Based on these observations, in this paper, we design Optum, a unified data center scheduler for improving the overall resource utilization while ensuring good performance for each application. Optum formulates an optimization problem to schedule unified task requests, aiming to balance the trade-off between utilization and resource contention. Optum also implements efficient heuristics to solve the optimization problem in a scalable manner. Large-scale experiments demonstrate that Optum can save up to 15% of resources without performance degradation compared to state-of-the-art unified scheduling schemes.
- 8 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 416-432
Open AccessUnderstanding and Optimizing Workloads for Unified Resource Management in Large Cloud Platforms
To fully utilize computing resources, cloud providers such as Google and Alibaba choose to co-locate online services with batch processing applications in their data centers. By implementing unified resource management policies, different types of complex computing jobs request resources in a consistent way, which can help data centers achieve global optimal scheduling and provide computing power with higher quality. To understand this new scheduling paradigm, in this paper, we first present an in-depth study of Alibaba's unified scheduling workloads. Our study focuses on the characterization of resource utilization, the application running performance, and scheduling scalability. We observe that although computing resources are significantly over-committed under unified scheduling, the resource utilization in Alibaba data centers is still low. In addition, existing resource usage predictors tend to make severe overestimations. At the same time, tasks within the same application behave fairly consistently, and the running performance of tasks can be well-profiled with respect to resource contention on the corresponding physical host. Based on these observations, in this paper, we design Optum, a unified data center scheduler for improving the overall resource utilization while ensuring good performance for each application. Optum formulates an optimization problem to schedule unified task requests, aiming to balance the trade-off between utilization and resource contention. Optum also implements efficient heuristics to solve the optimization problem in a scalable manner. Large-scale experiments demonstrate that Optum can save up to 15% of resources without performance degradation compared to state-of-the-art unified scheduling schemes.