SKV-Cache: A Topology-Aware Cache Framework for Efficient Question Answering over Governmental Archival KGs

Authors:

Abstract
In an era where generative AI is rapidly transforming how humans interact with structured data, reasoning over knowledge graphs (KGs) remains a computational bottleneck—especially in high-stakes domains such as legal and archival systems. Multi-hop question answering over regulatory knowledge graphs demands both semantic depth and execution speed. However, conventional Key-Value (KV) caching mechanisms in Large Language Models (LLMs) are designed for linear token streams and fail to leverage the graph-structured semantics inherent in KG traversal. This inefficiency leads to substantial inference latency, especially as reasoning paths deepen. To address this, we propose SKV-Cache: a Semantic Key-Value Cache that aligns LLM inference with KG topologies through semantic prefetching, relation-aware cache organization, and dynamic reuse of intermediate reasoning states. SKV-Cache anticipates and precomputes likely next-hop entities, enabling faster and context-aware inference. Experiments on a custom Dynamic Archives KG benchmark demonstrate that SKV-Cache achieves 2.14× faster inference than standard KV caching, with up to 89% cache hit rates on 3-hop queries—all while maintaining comparable answer accuracy. Our findings show that semantic-aware caching is not just an optimization, but a foundational shift for scalable, real-time KGQA. This opens new directions for responsive AI assistants in archival, legal, and public-sector information systems.