EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learning
Preprint
Abstract
Text-based games (TBGs) have emerged as an important collection of NLP tasks, requiring reinforcement learning (RL) agents to combine natural language understanding with reasoning. A key challenge for agents attempting to solve such tasks is to generalize across multiple games and demonstrate good performance on both seen and unseen objects. Purely deep-RL-based approaches may perform well on seen objects; however, they fail to showcase the same performance on unseen objects. Commonsense-infused deep-RL agents may work better on unseen data; unfortunately, their policies are often not interpretable or easily transferable. To tackle these issues, in this paper, we present EXPLORER which is an exploration-guided reasoning agent for textual reinforcement learning. EXPLORER is neurosymbolic in nature, as it relies on a neural module for exploration and a symbolic module for exploitation. It can also learn generalized symbolic policies and perform well over unseen data. Our experiments show that EXPLORER outperforms the baseline agents on Text-World cooking (TW-Cooking) and Text-World Commonsense (TWC) games.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning2023
- An ASP-based Approach to Answering Natural Language Questions for TextsTheory and Practice of Logic Programming, 2022
- Justifications for Goal-Directed Constraint Answer Set ProgrammingElectronic Proceedings in Theoretical Computer Science, 2020
- TextWorld: A Learning Environment for Text-Based Games2019
- Constraint Answer Set Programming without GroundingTheory and Practice of Logic Programming, 2018
- PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making2018
- Learning Explanatory Rules from Noisy DataJournal of Artificial Intelligence Research, 2018
- A new algorithm to automate inductive learning of default theoriesTheory and Practice of Logic Programming, 2017
- Language Understanding for Text-based Games using Deep Reinforcement Learning2015
- Nonmonotonic abductive inductive learningJournal of Applied Logic, 2009