KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 19474-19488
Abstract
Chaoyi Jiang, Lei Gao, Hossein Entezari Zarch, Murali Annavaram. Findings of the Association for Computational Linguistics: ACL 2025. 2025.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference2024
- CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving2024
- ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching2024
- Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society2023
- Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)2022