zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
Preprint
Abstract
Modeling evolving knowledge over temporal knowledge graphs (TKGs) has become a heated topic. Various methods have been proposed to forecast links on TKGs. Most of them are embedding-based, where hidden representations are learned to represent knowledge graph (KG) entities and relations based on the observed graph contexts. Although these methods show strong performance on traditional TKG forecasting (TKGF) benchmarks, they face a strong challenge in modeling the unseen zero-shot relations that have no prior graph context. In this paper, we try to mitigate this problem as follows. We first input the text descriptions of KG relations into large language models (LLMs) for generating relation representations, and then introduce them into embedding-based TKGF methods. LLM-empowered representations can capture the semantic information in the relation descriptions. This makes the relations, whether seen or unseen, with similar semantic meanings stay close in the embedding space, enabling TKGF models to recognize zero-shot relations even without any observed graph context. Experimental results show that our approach helps TKGF models to achieve much better performance in forecasting the facts with previously unseen relations, while still maintaining their ability in link forecasting regarding seen relations.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Improving Few-Shot Inductive Learning on Temporal Knowledge Graphs Using Confidence-Augmented Reinforcement Learning2023
- Comparing Apples and Oranges? On the Evaluation of Methods for Temporal Knowledge Graph Forecasting2023
- Incorporating Structured Sentences with Time-enhanced BERT for Fully-inductive Temporal Relation Prediction2023
- Learning Meta-Representations of One-shot Relations for Temporal Knowledge Graph Link Prediction2023
- Meta-Learning Based Knowledge Extrapolation for Temporal Knowledge Graph2023
- One-shot relational learning for extrapolation reasoning on temporal knowledge graphsData Mining and Knowledge Discovery, 2023
- RETIA: Relation-Entity Twin-Interact Aggregation for Temporal Knowledge Graph Extrapolation2023
- Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning2021
- Deriving Validity Time in Knowledge Graph2018
- Introducing ACLED: An Armed Conflict Location and Event DatasetJournal of Peace Research, 2010