NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time
Authors:
Open Access
- 1 January 2024
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 7913-7926
Abstract
Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, Hua Wu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding2024
- Effective Long-Context Scaling of Foundation Models2024
- RoFormer: Enhanced transformer with Rotary Position EmbeddingNeurocomputing, 2023
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints2023
- ERNIE-Doc: A Retrospective Long-Document Modeling Transformer2021
- QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization2021
- The NarrativeQA Reading Comprehension ChallengeTransactions of the Association for Computational Linguistics, 2018
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering2018
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering2018
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension2017