DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlog
Preprint
Abstract
Harvesting question-answer (QA) pairs from customer service chatlog in the wild is an efficient way to enrich the knowledge base for customer service chatbots in the cold start or continuous integration scenarios. Prior work attempts to obtain 1-to-1 QA pairs from growing customer service chatlog, which fails to integrate the incomplete utterances from the dialog context for composite QA retrieval. In this paper, we propose N-to-N QA extraction task in which the derived questions and corresponding answers might be separated across different utterances. We introduce a suite of generative/discriminative tagging based methods with end-to-end and two-stage variants that perform well on 5 customer service datasets and for the first time setup a benchmark for N-to-N DialogQAE with utterance and session level evaluation metrics. With a deep dive into extracted QA pairs, we find that the relations between and inside the QA pairs can be indicators to analyze the dialogue structure, e.g. information seeking, clarification, barge-in and elaboration. We also show that the proposed models can adapt to different domains and languages, and reduce the labor cost of knowledge accumulation in the real-world product dialogue platform.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarization2021
- PRIDE: Predicting Relationships in Conversations2021
- Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker2021
- Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation2021
- A Dialogue-based Information Extraction System for Medical Insurance Assessment2021
- Structured Attention for Unsupervised Dialogue Structure Induction2020
- Transformers: State-of-the-Art Natural Language Processing2020
- Dialogue-Based Relation Extraction2020
- Automatic extraction of personal events from dialogue2020
- A Multi-Agent Communication Framework for Question-Worthy Phrase Extraction and Question GenerationProceedings of the AAAI Conference on Artificial Intelligence, 2019