Harmonia: QoS-Aware and High-Throughput Generative Inference with a Single GPU
- 15 August 2026
- book chapter
- Published by Springer Nature
- p. 363-378
Abstract
No abstract availableKeywords
This publication has 12 references indexed in Scilit:
- Aegaeon: Effective GPU Pooling for Concurrent LLM Serving on the Market2025
- Mercury: Unlocking Multi-GPU Operator Optimization for LLMs via Remote Memory Scheduling2025
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs2025
- Performance Characterization of Expert Router for Scalable LLM Inference2024
- LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services2024
- Improving Throughput-oriented LLM Inference with CPU Computations2024
- Understanding Performance Implications of LLM Inference on CPUs2024
- An Introduction to the Compute Express Link (CXL) InterconnectACM Computing Surveys, 2024
- Exploring Performance and Cost Optimization with ASIC-Based CXL Memory2024
- DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale2022