Towards Structure-Aware Model for Multi-Modal Knowledge Graph Completion
- 3 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1404-1416
Abstract
Knowledge graphs (KGs) play a key role in promoting various multimedia and AI applications. However, with the explosive growth of multi-modal information, traditional knowledge graph completion (KGC) models cannot be directly applied. This has attracted a large number of researchers to study multi-modal knowledge graph completion (MMKGC). Since MMKG extends KG to the visual and textual domains, MMKGC faces two main challenges: (1) how to deal with the fine-grained modality information interaction and awareness; (2) how to ensure the dominant role of graph structure in multi-modal knowledge fusion and deal with the noise generated by other modalities during modality fusion. To address these challenges, this paper proposes a novel MMKGC model named TSAM, which integrates fine-grained modality interaction and dominant graph structure to form a high-performance MMKGC framework. Specifically, to solve the challenges, TSAM proposes the Fine-grained Modality Awareness Fusion method (FgMAF), which uses pre-trained language models better to capture fine-grained semantic information interaction of different modalities and employs an attention mechanism to achieve fine-grained modality awareness and fusion. Additionally, TSAM presents the Structure-aware Contrastive Learning method (SaCL), which utilizes two contrastive learning approaches to align other modalities more closely with the structured modality. Extensive experiments show the proposed TSAM model significantly outperforms existing MMKGC models on widely used multi-modal datasets. The code is available at https://github.com/2391134843/TSAM.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62436006)
This publication has 57 references indexed in Scilit:
- Knowledge Graphs for Multi-modal Learning: Survey and PerspectiveInformation Fusion, 2025
- Embracing Unimodal Aleatoric Uncertainty for Robust Multimodal Fusion2024
- Multi-Modal Siamese Network for Few-Shot Knowledge Graph Completion2024
- Knowledge graph completion method based on quantum embedding and quaternion interaction enhancementInformation Sciences, 2023
- MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoning2023
- Relational Message Passing for Fully Inductive Knowledge Graph Completion2023
- A knowledge graph completion model based on contrastive learning and relation enhancement methodKnowledge-Based Systems, 2022
- Understanding the Behaviour of Contrastive Loss2021
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- MMKG: Multi-modal Knowledge Graphs2019