Learning Multi-turn Response Selection in Grounded Dialogues with Reinforced Knowledge and Context Distillation
Open Access
- 21 April 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 41 (4) , 1-27
Abstract
Recently, knowledge-grounded dialogue systems have gained increasing attention. Great efforts have been made to build response matching models where all dialogue content and knowledge sentences are leveraged. However, knowledge redundancy and distraction of irrelevant dialogue content often exist in knowledge-grounded conversations, which may affect the matching process and lead to inferior performance. In addition, irrelevant dialogue history and excessive knowledge also hinder the exploitation of popular pre-trained language models (PLMs) due to the limitation of input length. To address these challenges, we propose a new knowledge-grounded dialogue model based on PLMs, where a knowledge selector and a context selector are designed for filtering out irrelevant knowledge sentences and redundant dialogue history, respectively. Considering the lack of labeled data for the learning of two selectors, we pre-train them with weakly-supervised tasks and then jointly conduct the optimization of knowledge and context selection and fine-tuning of PLMs for response ranking with reinforcement learning (RL). By this means, the dialogue model can distill more accurate and concise knowledge and dialogue content for subsequent response ranking module, and the overall model can converge and perform better. We conduct experiments on two benchmarks and evaluation results indicate that our model can significantly outperform the state-of-the-art methods.Keywords
This publication has 10 references indexed in Scilit:
- Adapting to Context-Aware Knowledge in Natural Conversation for Multi-Turn Response Selection2021
- Training Neural Response Selection for Task-Oriented Dialogue Systems2019
- From Eliza to XiaoIce: challenges and opportunities with social chatbotsFrontiers of Information Technology & Electronic Engineering, 2018
- Training Millions of Personalized Dialogue Agents2018
- A Dataset for Document Grounded Conversations2018
- Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System2016
- Deep Reinforcement Learning for Mention-Ranking Coreference Models2016
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems2015
- ELIZA—a computer program for the study of natural language communication between man and machineCommunications of the ACM, 1966