Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
Authors:
Top Cited Papers
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 7655-7671
Abstract
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, Zhifang Sui. Findings of the Association for Computational Linguistics ACL 2024. 2024.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Multi-candidate Speculative Decoding2025
- SPEED: Speculative Pipelined Execution for Efficient Decoding2025
- SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Grammatical Error Correction: A Survey of the State of the ArtComputational Linguistics, 2023
- A Comprehensive Survey of Grammatical Error CorrectionACM Transactions on Intelligent Systems and Technology, 2021
- Dense Passage Retrieval for Open-Domain Question Answering2020
- Natural Questions: A Benchmark for Question Answering ResearchTransactions of the Association for Computational Linguistics, 2019
- Latency lags bandwithCommunications of the ACM, 2004
- Speculative computation, parallelism, and functional programmingIEEE Transactions on Computers, 1985