Dictionary Multi-Modal Temporal Graph Learning
- 31 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (8) , 9606-9623
Abstract
Temporal graph learning focuses on graph deep learning in real-world dynamic scenarios, which uses interaction sequence instead of adjacency matrix to observe the graph dynamic changes more microscopically from the perspective of time evolution. However, current temporal graph methods only focus on extra dynamic information, ignoring the large amount of multi-modal information contained in the real world. These information can reflect the rich changes in the real world from different perspectives. Ignoring them means that temporal graph learning still lacks the ability to restore and mine more complex real-world data. We argue that the main challenges causing the above phenomenon in temporal graph learning are the lack of multi-modal architecture and public multi-modal datasets. To solve the above challenges, we propose ModalTGL, which enhances the computational efficiency of the model in complex dynamic scenarios by introducing the dictionary graph network, and achieves multi-modal fusion by embedding tuning. In addition, we also discuss the effects of different time encoding functions on dynamic information preservation. At the data level, we build several multi-modal temporal graph datasets from different areas, and compare with multiple SOTA methods on these datasets. The experimental results verify the effectiveness of the ModalTGL method, achieving the performance improvement of up to 18.48%. Code and data can be obtained from https://github.com/MGitHubL/ModalTGL.Keywords
Funding Information
- National Natural Science Foundation of China for Distinguished Young Scholar (62325604)
- Major Program Project of Xiangjiang Laboratory (24XJJCYJ01002)
- National Natural Science Foundation of China (62276271)
- National Natural Science Foundation of China (62506371)
- National Natural Science Foundation of China (62441618)
This publication has 57 references indexed in Scilit:
- Rethinking multi-level information fusion in temporal graphs: Pre-training then distilling for better embeddingInformation Fusion, 2025
- MetaQ: fast, scalable and accurate metacell inference via single-cell quantizationNature Communications, 2025
- Contrastive cross-modal clustering with twin networkPattern Recognition, 2024
- PDDGCN: A Parasitic Disease–Drug Association Predictor Based on Multi-view Fusion Graph Convolutional NetworkInterdisciplinary Sciences: Computational Life Sciences, 2024
- Predicting Potential Drug–Disease Associations Based on Hypergraph Learning with Subgraph MatchingInterdisciplinary Sciences: Computational Life Sciences, 2023
- MST-GAT: A multimodal spatial–temporal graph attention network for time series anomaly detectionInformation Fusion, 2022
- Twin Contrastive Learning for Online ClusteringInternational Journal of Computer Vision, 2022
- Multi-graph fusion for multi-view spectral clusteringKnowledge-Based Systems, 2020
- Discriminative graph regularized broad learning system for image recognitionScience China Information Sciences, 2018
- The dynamic functional connectome: State-of-the-art and perspectivesNeuroImage, 2017