Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines
Preprint
Abstract
Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions, and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- LeDeepChef Deep Reinforcement Learning Agent for Families of Text-Based GamesProceedings of the AAAI Conference on Artificial Intelligence, 2020
- TextWorld: A Learning Environment for Text-Based Games2019
- ConceptNet 5.5: An Open Multilingual Graph of General KnowledgeProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Transferring knowledge as heuristics in reinforcement learning: A case-based approachArtificial Intelligence, 2015
- Commonsense reasoning and commonsense knowledge in artificial intelligenceCommunications of the ACM, 2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- Language Understanding for Text-based Games using Deep Reinforcement Learning2015
- Learning to Win by Reading Manuals in a Monte-Carlo FrameworkJournal of Artificial Intelligence Research, 2012
- ConceptNet — A Practical Commonsense Reasoning Tool-KitBT Technology Journal, 2004
- Planning and acting in partially observable stochastic domainsArtificial Intelligence, 1998