I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
Authors:
- Pei ZhouPZPei Zhou
- Andrew ZhuAZAndrew Zhu
- Jennifer HuJHJennifer Hu
- Jay PujaraJPJay Pujara
- Prithviraj AmmanabroluPAPrithviraj Ammanabrolu
Preprint
Abstract
We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment. Dungeons and Dragons (D&D), a role-playing game, provides an ideal setting to investigate such interactions. Here, the Dungeon Master (DM), i.e., the teacher, guides the actions of several players -- students, each with their own personas and abilities -- to achieve shared goals grounded in a fantasy world. Our approach is to decompose and model these interactions into (1) the DM's intent to guide players toward a given goal; (2) the DM's guidance utterance to the players expressing this intent; and (3) a theory-of-mind (ToM) model that anticipates the players' reaction to the guidance one turn into the future. We develop a novel reinforcement learning (RL) method for training a DM that generates guidance for players by rewarding utterances where the intent matches the ToM-anticipated player actions. Human and automated evaluations show that a DM trained to explicitly model intents and incorporate ToM of the players using RL generates better-quality guidance that is 3x more likely to fulfill the DM's intent than a vanilla natural language generation (NLG) approach.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals2023
- Human-level play in the game ofDiplomacyby combining language models with strategic reasoningScience, 2022
- Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs2022
- Learning to Speak and Act in a Fantasy Text Adventure Game2019
- Does the chimpanzee have a theory of mind?Behavioral and Brain Sciences, 1978