Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
Authors:
Top Cited Papers
- 18 May 2026
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-10
Abstract
No abstract availableAll Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 22 references indexed in Scilit:
- Variance-Aware Bandit Framework for Dynamic Probabilistic Maximum Coverage Problem With Triggered or Self-Reliant ArmsIEEE Transactions on Networking, 2025
- Survey of vector database management systemsThe VLDB Journal, 2024
- SCALM: Towards Semantic Caching for Automated Chat Services with Large Language Models2024
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference2023
- Similarity Caching: Theory and Algorithms2020
- Similarity caching2009
- Nearest-neighbor caching for content-match applications2009
- Web cache management based on the expected cost of web objectsInformation and Software Technology, 2005
- LRFU: a spectrum of policies that subsumes the least recently used and least frequently used policiesIEEE Transactions on Computers, 2001
- An approximation guarantee of the greedy descent algorithm for minimizing a supermodular set functionDiscrete Applied Mathematics, 2001