Learning to Generate Prompts for Dialogue Generation through Reinforcement Learning
Preprint
Abstract
Much literature has shown that prompt-based learning is an efficient method to make use of the large pre-trained language model. Recent works also exhibit the possibility of steering a chatbot's output by plugging in an appropriate prompt. Gradient-based methods are often used to perturb the prompts. However, some language models are not even available to the public. In this work, we first explored the combination of prompting and reinforcement learning (RL) to steer models' generation without accessing any of the models' parameters. Second, to reduce the training effort and enhance the generalizability to the unseen task, we apply multi-task learning to make the model learn to generalize to new tasks better. The experiment results show that our proposed method can successfully control several state-of-the-art (SOTA) dialogue models without accessing their parameters. Furthermore, the model demonstrates the strong ability to quickly adapt to an unseen task in fewer steps than the baseline model.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- GeDi: Generative Discriminator Guided Sequence Generation2021
- Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference2021
- Recipes for Building an Open-Domain Chatbot2021
- DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts2021
- Put Chatbot into Its Interlocutor’s Shoes: New Framework to Learn Chatbot Responding with Intention2021
- Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning2020
- Towards Controllable Biases in Language Generation2020
- DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation2020
- Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills2020
- Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue2019