From Concept to Instance: Hierarchical Reinforced Knowledge Graph Reasoning
- 27 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 8 (5) , 3665-3677
Abstract
The knowledge graph, a networked structure designed to organize the vast and heterogeneous knowledge existing in the real world, has gained widespread adoption as a background knowledge base for intelligent systems. Nevertheless, the incompleteness of knowledge graphs has been widely recognized as a significant challenge in their development and application. Recently, multi-hop knowledge graph reasoning has been an attractive method for completing a knowledge graph. The multi-hop knowledge graph reasoning based on the reinforcement learning (RL) framework has achieved promising performance in terms of interpretability and scalability. An RL agent automatically reasons over a KG under the guidance of a policy network. When faced with a query, obtaining the approximate range of the answer first and then delving into individual options is a more efficient approach compared to traversing all candidate answers. However, existing RL-based methods have two limitations. First, they lack the ability to filter candidate decisions, making it challenging to handle entities with a large number of neighbors. Second, they fail to consider the intrinsic correlations between entities. To address these limitations, we propose a novel hierarchical knowledge graph reasoning approach HiKGR , which leverages the concept information of entities. Specifically, HiKGR reconstructs the previous action space in RL into a concept space and an instance space, enabling two policies to alternate reasoning at the concept level and the instance level. Furthermore, we propose hierarchical reward functions for the two-level policies to achieve joint optimization. The hierarchical reasoning approach we propose is capable of selecting more reasonable candidate decisions and optimizing the decision space. Experimental results reveal that HiKGR significantly outperforms existing RL-based methods and drastically reduces the action space size.Keywords
Funding Information
- National Key R&D Program of China (2023YFF0905503)
- National Natural Science Foundation of China (62072203)
This publication has 34 references indexed in Scilit:
- HyGGE: Hyperbolic graph attention network for reasoning over knowledge graphsInformation Sciences, 2023
- A Game-Based Evolutionary Clustering With Historical Information Aggregation for Personal RecommendationIEEE Transactions on Emerging Topics in Computational Intelligence, 2022
- Step by step: A hierarchical framework for multi-hop knowledge graph reasoning with reinforcement learningKnowledge-Based Systems, 2022
- Counterfactual-Guided and Curiosity-Driven Multi-hop Reasoning over Knowledge Graph2022
- Reinforcement Learning-Based Optimization for Mobile Edge Computing Scheduling GameIEEE Transactions on Emerging Topics in Computational Intelligence, 2022
- Dynamic knowledge graph reasoning based on deep reinforcement learningKnowledge-Based Systems, 2022
- HOPLoP: multi-hop link prediction over knowledge graph embeddingsWorld Wide Web, 2021
- Improving Knowledge Graph Embeddings with Ontological Reasoning2021
- Frequent Pattern Mining in Big Social GraphsIEEE Transactions on Emerging Topics in Computational Intelligence, 2021
- Strong Social Graph Based Trust-Oriented Graph Pattern Matching With Multiple ConstraintsIEEE Transactions on Emerging Topics in Computational Intelligence, 2020