High Throughput and Low Latency LLM Serving via Adaptive KV Caching
Open Access
- 26 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 56-71
Abstract
No abstract availableFunding Information
- Science and Technology Development Fund of Macau (0041/2025/RIA1)
- Science and Technology Development Fund of Macau (0074/2025/AMJ)
- Multi-Year Research Grant of University of Macau (MYRG-GRG2024-00255-FST-UMDF)
- Multi-Year Research Grant of University of Macau (MYRG-GRG2025-00119-FST)
- Guangdong Basic and Applied Basic Research Foundation (No. 2023B1515130002)
- Alibaba Group through Alibaba Innovative Research Program (48817586)
This publication has 19 references indexed in Scilit:
- DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency2025
- Splitwise: Efficient Generative LLM Inference Using Phase Splitting2024
- Optimal Kernel Orchestration for Tensor Programs with Korch2024
- Optimizing Deep Learning Inference via Global Analysis and Tensor Expressions2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion2023
- DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale2022
- Automatic Horizontal Fusion for GPU Kernels2022
- Author Correction: SciPy 1.0: fundamental algorithms for scientific computing in PythonNature Methods, 2020
- Sequential Quadratic Programming Methods2011