Continuous Semantic Caching for Low-Cost LLM Serving
Open Access
- 22 November 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 381-390
Abstract
No abstract availableThis publication has 10 references indexed in Scilit:
- Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation2026
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference2023
- Towards Optimal Caching and Model Selection for Large Model Inference2023
- GPTCache: An Open-Source Semantic Cache for LLM Applications Enabling Faster Answers and Cost Savings2023
- Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models2023
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks2019
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension2017
- Web cache management based on the expected cost of web objectsInformation and Software Technology, 2005
- An approximation guarantee of the greedy descent algorithm for minimizing a supermodular set functionDiscrete Applied Mathematics, 2001