Archit Patke
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Archit Patke has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Ravishankar K. Iyer
University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
3 shared publications - Saurabh Jha
IBM Research, Yorktown Heights, New York, USA
3 shared publications - Zbigniew Kalbarczyk
University of Illinois at Urbana-Champaign, Illinois, USA
3 shared publications - Chandra Narayanaswami
IBM Research, Yorktown Heights, NY, USA
2 shared publications - Christian Pinto
IBM Research Europe, Dublin, Ireland
2 shared publications - Haoran Qiu
University of Illinois at Urbana-Champaign, Urbana, USA
2 shared publications
Latest Publications
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1145-1164
Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1145-1164
Open AccessStory of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs
- Open Access
- 8 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1206-1218
Page Migration for Hardware Memory Disaggregation Across a Network
- 8 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1206-1218
Open AccessPage Migration for Hardware Memory Disaggregation Across a Network
- Open Access
- 20 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 18-35
Queue Management for SLO-Oriented Large Language Model Serving
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots and coding assistants, with tight latency SLO requirements. However, when such systems execute batch requests that have relaxed SLOs along with interactive requests, it leads to poor multiplexing and inefficient resource utilization. To address these challenges, we propose QLM, a queue management system for LLM serving. QLM maintains batch and interactive requests across different models and SLOs in a request queue. Optimal ordering of the request queue is critical to maintain SLOs while ensuring high resource utilization. To generate this optimal ordering, QLM uses a Request Waiting Time (RWT) Estimator that estimates the waiting times for requests in the request queue. These estimates are used by a global scheduler to orchestrate LLM Serving Operations (LSOs) such as request pulling, request eviction, load balancing, and model swapping. Evaluation on heterogeneous GPU devices and models with real-world LLM serving dataset shows that QLM improves SLO attainment by 40-90% and throughput by 20-400% while maintaining or improving device utilization compared to other state-of-the-art LLM serving systems. QLM's evaluation is based on the production requirements of a cloud provider. QLM is publicly available at https://www.github.com/QLM-project/QLM.
- 20 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 18-35
Open AccessQueue Management for SLO-Oriented Large Language Model Serving
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots and coding assistants, with tight latency SLO requirements. However, when such systems execute batch requests that have relaxed SLOs along with interactive requests, it leads to poor multiplexing and inefficient resource utilization. To address these challenges, we propose QLM, a queue management system for LLM serving. QLM maintains batch and interactive requests across different models and SLOs in a request queue. Optimal ordering of the request queue is critical to maintain SLOs while ensuring high resource utilization. To generate this optimal ordering, QLM uses a Request Waiting Time (RWT) Estimator that estimates the waiting times for requests in the request queue. These estimates are used by a global scheduler to orchestrate LLM Serving Operations (LSOs) such as request pulling, request eviction, load balancing, and model swapping. Evaluation on heterogeneous GPU devices and models with real-world LLM serving dataset shows that QLM improves SLO attainment by 40-90% and throughput by 20-400% while maintaining or improving device utilization compared to other state-of-the-art LLM serving systems. QLM's evaluation is based on the production requirements of a cloud provider. QLM is publicly available at https://www.github.com/QLM-project/QLM.