Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning
Preprint
Abstract
We present a two-step hybrid reinforcement learning (RL) policy that is designed to generate interpretable and robust hierarchical policies on the RL problem with graph-based input. Unlike prior deep reinforcement learning policies parameterized by an end-to-end black-box graph neural network, our approach disentangles the decision-making process into two steps. The first step is a simplified classification problem that maps the graph input to an action group where all actions share a similar semantic meaning. The second step implements a sophisticated rule-miner that conducts explicit one-hop reasoning over the graph and identifies decisive edges in the graph input without the necessity of heavy domain knowledge. This two-step hybrid policy presents human-friendly interpretations and achieves better performance in terms of generalization and robustness. Extensive experimental studies on four levels of complex text-based games have demonstrated the superiority of the proposed method compared to the state-of-the-art.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Learning to Act with Affordance-Aware Multimodal Neural SLAM2022
- Graphhopper: Multi-hop Scene Graph Reasoning for Visual Question Answering2021
- A graph placement methodology for fast chip designNature, 2021
- Mastering Atari, Go, chess and shogi by planning with a learned modelNature, 2020
- Machine Learning Methods in Drug DiscoveryMolecules, 2020
- A Mixed approach of Deep Learning method and Rule-Based method to improve Aspect Level Sentiment AnalysisApplied Computing and Informatics, 2020
- Stop explaining black box machine learning models for high stakes decisions and use interpretable models insteadNature Machine Intelligence, 2019
- Mining Frequent Patterns without Candidate Generation: A Frequent-Pattern Tree ApproachData Mining and Knowledge Discovery, 2004
- Combining a neural network with a rule-based expert system approach for short-term power load forecasting in TaiwanExpert Systems with Applications, 1997