Contextual Knowledge Infusion via Iterative Semantic Tracing for Vision–Language Understanding
Preprint
- 14 November 2025
- preprint
- Published by MDPI AG in Preprints.org
Abstract
The challenge of integrating external knowledge into visual reasoning frameworks has motivated a growing interest in models capable of bridging perceptual understanding with abstract, non-visual information. Unlike conventional visual question answering settings, knowledge-driven VQA demands a joint interpretation of visible cues and facts that are absent from the image itself. This paper introduces a new perspective on this task and proposes \textsc{KV-Trace}, a unified semantic tracing framework that emphasizes iterative knowledge refinement and structured visual interpretation. Instead of treating visual and knowledge modalities as homogeneous sources, our framework explicitly distinguishes their representational roles and organizes them into a progressive reasoning pipeline. Through a dynamic knowledge memory space and a query-sensitive semantic propagation mechanism, \textsc{KV-Trace} composes multi-stage reasoning steps that evolve according to the underlying question. Extensive experiments conducted on the KRVQR and FVQA benchmarks demonstrate that our model achieves improved reasoning depth and generalization capacity. Additional ablation studies further verify the contribution of each reasoning component and highlight the interpretability benefits gained from explicit knowledge structuring.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Knowledge-enhanced event relation extraction via event ontology promptInformation Fusion, 2023
- Cross-modal knowledge reasoning for knowledge-based visual question answeringPattern Recognition, 2020
- A span-graph neural model for overlapping entity relation extraction in biomedical textsBioinformatics, 2020
- Boundaries and edges rethinking: An end-to-end neural model for overlapping entity relation extractionInformation Processing & Management, 2020
- Enriching contextualized language model from knowledge graph for biomedical information extractionBriefings in Bioinformatics, 2020
- Dispatched attention with multi-task learning for nested mention recognitionInformation Sciences, 2020
- A tree-based neural network model for biomedical event trigger detectionInformation Sciences, 2020
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- Deep learningNature, 2015
- A sensor for dynamic tactile information with applications in human–robot interaction and object explorationRobotics and Autonomous Systems, 2006