When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation
Open Access
- 12 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3859-3869
Abstract
Knowledge Graph Retrieval-Augmented Generation (KG-RAG) extends the RAG paradigm by incorporating structured knowledge from knowledge graphs, enabling Large Language Models (LLMs) to perform more precise and explainable reasoning. While KG-RAG improves factual accuracy in complex tasks, existing KG-RAG models are often severely overconfident, producing high-confidence predictions even when retrieved sub-graphs are incomplete or unreliable, which raises concerns for deployment in high-stakes domains. To address this issue, we propose Ca2KG, a Causality-aware Calibration framework for KG-RAG. Ca2KG integrates counterfactual prompting, which exposes retrieval-dependent uncertainties in knowledge quality and reasoning reliability, with a panel-based re-scoring mechanism that stabilises predictions across interventions. Extensive experiments on two complex QA datasets demonstrate that Ca2KG consistently improves calibration while maintaining or even enhancing predictive accuracy. The source code can be found at~ https://aisuko.github.io/ca2kg/.Keywords
This publication has 32 references indexed in Scilit:
- Unbiased Reasoning for Knowledge-Intensive Tasks in Large Language Models via Conditional Front-Door Adjustment2025
- A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future DirectionsACM Computing Surveys, 2025
- Data-driven learning optimal K values for K-nearest neighbour matching in causal inferenceData Mining and Knowledge Discovery, 2025
- KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation2025
- Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning2025
- What large language models know and what people think they knowNature Machine Intelligence, 2025
- Disentangled Representation Learning for Causal Inference With InstrumentsIEEE Transactions on Neural Networks and Learning Systems, 2024
- Evaluating Retrieval Quality in Retrieval-Augmented Generation2024
- A Survey on Evaluation of Large Language ModelsACM Transactions on Intelligent Systems and Technology, 2024
- Graph Learning for Anomaly Analytics: Algorithms, Applications, and ChallengesACM Transactions on Intelligent Systems and Technology, 2023