Diagnosing Disagreement Observability Collapse in LLM-based Structured KnowledgeVerification: Restoring Uncertainty Signals through Task-Aligned Multi-View Verification
Authors:
- Zhehao LiangZLZhehao Liang
- Lu LiuLLLu Liu
- Tao PengTPTao Peng
Preprint
- 22 September 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Large language models are increasingly used for structured knowledge verification, where reliable knowledge verification is essential for identifying potentially incorrect predictions. Existing disagreement-based uncertainty methods assume that repeated outputspreserve observable variation. However, semantically plausible false claims may produce unanimous but incorrect predictions, causing disagreement observability collapse before uncertainty computation. On a controlled stress test, SC-3 has Zero-Entropy Ratesof 83.9% on deceptive examples and 76.5% on 1,491 aligned examples. Its Wrong-Unanimity Rate (WUR) is 67.5% (326/483 errors). As a diagnostic intervention, Multi-View Conflict Aggregation (MVCA) decomposes verification into factual, semantic,and structural dimensions rather than introducing arbitrary prompt diversity. MVCA reduces the corresponding Zero-Entropy Rates to 39.1% and 55.9% and has a WUR of 39.1% (175/448 errors). It obtains AURC 0.234 with three calls, compared with 0.231 for 20 homogeneous samples. The benefit reverses on weaker backbones that do not respond differently to the assigned dimensions. Reliable LLM-based knowledge verification depends on whether diagnostically useful disagreement remains observable before uncertainty aggregation. Task-aligned verification decomposition helps only when the backbone can distinguish the relevant verification dimensions.Keywords
This publication has 12 references indexed in Scilit:
- Bridging language models and knowledge graphs with controlled natural languagesKnowledge-Based Systems, 2026
- Normalizing flow-enhanced Gaussian embedding for few-shot knowledge graph completionKnowledge-Based Systems, 2025
- Prompting large language models with knowledge graphs for question answering involving long-tail factsKnowledge-Based Systems, 2025
- AI models collapse when trained on recursively generated dataNature, 2024
- Enhancing text-based knowledge graph completion with zero-shot large language models: A focus on semantic enhancementKnowledge-Based Systems, 2024
- Detecting hallucinations in large language models using semantic entropyNature, 2024
- Survey of Hallucination in Natural Language GenerationACM Computing Surveys, 2023
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models2023
- Knowledge structures in a tolerance knowledge base and their uncertainty measuresKnowledge-Based Systems, 2018
- Knowledge Graph Embedding: A Survey of Approaches and ApplicationsIEEE Transactions on Knowledge and Data Engineering, 2017