Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach
Preprint
Abstract
Despite numerous successes in Deep Reinforcement Learning (DRL), the learned policies are not interpretable. Moreover, since DRL does not exploit symbolic relational representations, it has difficulties in coping with structural changes in its environment (such as increasing the number of objects). Relational Reinforcement Learning, on the other hand, inherits the relational representations from symbolic planning to learn reusable policies. However, it has so far been unable to scale up and exploit the power of deep neural networks. We propose Deep Explainable Relational Reinforcement Learning (DERRL), a framework that exploits the best of both -- neural and symbolic worlds. By resorting to a neuro-symbolic approach, DERRL combines relational representations and constraints from symbolic planning with deep learning to extract interpretable policies. These policies are in the form of logical rules that explain how each decision (or action) is arrived at. Through several experiments, in setups like the Countdown Game, Blocks World, Gridworld, and Traffic, we show that the policies learned by DERRL can be applied to different configurations and contexts, hence generalizing to environmental modifications.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- T-norms driven loss functions for machine learningApplied Intelligence, 2023
- Deep Reinforcement Learning: A Brief SurveyIEEE Signal Processing Magazine, 2017
- Relational reinforcement learning with guided demonstrationsArtificial Intelligence, 2017
- Survey of Clustering AlgorithmsIEEE Transactions on Neural Networks, 2005
- Integrating Guidance into Relational Reinforcement LearningMachine Learning, 2004
- Monoidal t-norm based logictowards a logic for left-continuous t-normsFuzzy Sets and Systems, 2001
- Inductive Logic Programming: Theory and methodsThe Journal of Logic Programming, 1994
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- Learning Logical Definitions from RelationsMachine Learning, 1990
- Strips: A new approach to the application of theorem proving to problem solvingArtificial Intelligence, 1971