SPShijie Peng

Shijie Peng

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shijie Peng has a total of 13 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    ReliefServe: Relieving GPU Pressure in Multi-Model Serving via Selective CPU Escape

    by Shijie Peng, Yanying Lin, Chengzhi Lu, Shuaipeng Wu, Chengzhong Xu, Kejiang Ye

  • Workload-Adapted Resource Allocation for LLM Distributed Serving in Serverless Clusters

    by Yanying Lin, Shijie Peng, Yanbo Li, Shutian Luo, Haiying Shen, Kejiang Ye, Chengzhong Xu

    Large language models increasingly rely on pipeline parallelism for distributed inference, but existing systems face critical challenges in serverless environments: heterogeneous request distributions across pipeline stages and unpredictable workload patterns requiring rapid elasticity. Traditional static resource allocation fails to address pipeline-specific bottlenecks and cold start delays inherent in serverless architectures. We propose Quart, a workload-adapted resource allocation system for LLM distributed serving in serverless clusters. Quart introduces pipeline-aware resource management through: (1) latency-aware critical stage identification using coefficient of variation (CV)-based burst propagation analysis, (2) dynamic replica allocation with proportional-integral-derivative (PID) control for congested stages, and (3) hierarchical parameter caching with copy-on-write mechanisms enabling sub-second serverless scaling. The system addresses serverless-specific challenges through cache-aware scheduling that maintains model parameters in memory, eliminating disk I/O overhead during rapid scaling events. Evaluation with real-world workloads shows Quart reduces average response latency by up to 87.1% compared to existing serverless inference systems while achieving 2.37x improvement in goodput.

  • Serving LLM in Distributed GPU Cluster With Fine-Grain Pipeline Constraints

    by Yanying Lin, Shijie Peng, Shuaipeng Wu, Yanbo Li, Chengzhi Lu, Kejiang Ye, Chengzhong Xu

    As Large Language Models (LLMs) continue to advance, their parameter sizes are growing exponentially-far outpacing hardware capabilities. This widening gap necessitates distributed computing through pipeline parallelism for efficient inference. However, the uneven distribution of requests across pipeline stages creates significant performance bottlenecks in real-world deployments. To address this challenge, we present Planck, a performance optimization framework specifically designed for distributed LLM inference. Planck implements fine-grained control through two key mechanisms: a progressive SLO allocation strategy that dynamically adjusts time constraints based on workload patterns, and stage-specific performance controllers that prevent bottlenecks before they cascade through the system. By intelligently balancing resources across pipeline stages, Planck effectively eliminates queue buildup-essentially preventing traffic congestion before it forms. Evaluation using diverse workloads in real cloud environments demonstrates that Planck reduces P99 tail latency by up to 18% and decreases the longest queue lengths by as much as 47.8% across pipeline stages, significantly improving both system responsiveness and resource utilization.


  • Understanding Serverless Inference in Mobile-Edge Networks: A Benchmark Approach

    by Junhong Chen, Yanying Lin, Shijie Peng, Shuaipeng Wu, Kenneth Kent, Hao Dai, Kejiang Ye, Yang Wang

    Although the emerging serverless paradigm has the potential to become a dominant way of deploying cloud-service tasks across millions of mobile and IoT devices, the overhead characteristics of executing these tasks on such a volume of mobile devices remain largely unclear. To address this issue, this paper conducts a deep analysis based on the OpenFaaS platform—a popular open-source serverless platform for mobile edge environments—to investigate the overhead of performing deep learning inference tasks on mobile devices. To thoroughly evaluate the inference overhead, we develop a performance benchmark, named ESBench, whereby a set of comprehensive experiments are conducted with respect to a bunch of simulated mobile devices associated with an edge cluster. Our investigation reveals that the performance of deep learning inference tasks is significantly influenced by the model size and resource contention in mobile devices, leading to up to 3× degradation in performance. Moreover, we observe that the network environment can negatively impact the performance of mobile inference, increasing the CPU overhead under poor network conditions. Based on our findings, we further propose some recommendations for designing efficient serverless platforms and resource management strategies as well as for deploying serverless computing in the mobile edge environment.