Negative Training for Neural Dialogue Response Generation
Authors:
- Tianxing HeTHTianxing He
- James GlassJGJames Glass
Preprint
Abstract
Although deep learning models have brought tremendous advancements to the field of open-domain dialogue response generation, recent research results have revealed that the trained models have undesirable generation behaviors, such as malicious responses and generic (boring) responses. In this work, we propose a framework named "Negative Training" to minimize such behaviors. Given a trained model, the framework will first find generated samples that exhibit the undesirable behavior, and then use them to feed negative training signals for fine-tuning the model. Our experiments show that negative training can significantly reduce the hit rate of malicious responses (e.g. from 12.6% to 0%), or discourage frequent responses and improve response diversity (e.g. improve response entropy by over 63%).Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial ExamplesProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Deep Text Classification Can be Fooled2018
- ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations2018
- SeqGAN: Sequence Generative Adversarial Nets with Policy GradientProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Adversarial Learning for Neural Dialogue Generation2017
- Crafting adversarial input sequences for recurrent neural networks2016
- How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation2016
- The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- Minimum error rate training in statistical machine translation2003