AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving
Open Access
- 19 November 2025
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 361-374
Abstract
No abstract availableAll Related Versions
Funding Information
- the Science and Technology Commission of Shanghai Municipality (24DP1500704 and 24YL1901100)
- the Fundamental Research Funds for the Central Universities (22120230311)
- the Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence (2023B1212010001)
This publication has 12 references indexed in Scilit:
- Queue Management for SLO-Oriented Large Language Model Serving2024
- SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification2024
- LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages2024
- Draft& Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding2024
- Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Scrooge2021
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Low latency RNN inference with cellular batching2018