Using Synthetic Data to Train a Conversational Response Generation Model in Low Resource Settings
Authors:
- 27 October 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 306-311
Abstract
Response generation is a task in natural language processing where a model is trained to respond to human statements. Conversational response generators take this one step further with the ability to respond within the context of previous responses. While techniques exist for training such models, they all require large amounts of conversational data, which are not always available for low-resource languages. In this research, we make three contributions. First, we released the first Filipino conversational dataset collected from a popular Philippine online forum called the “PEx Conversations Dataset.” Second, we introduce a data augmentation methodology for Filipino data by employing a Tagalog RoBERTa model to increase the size of the existing corpora. Lastly, we published the first Filipino conversational response generator capable of generating responses related to the previous three responses. With the supplementary synthetic data, we were able to improve the performance of the response generator by up to 10.7% in perplexity, and 11.7% in content word usage as compared to training with zero synthetic data.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Training Question Answering Models From Synthetic Data2020
- Fluent Response Generation for Conversational Question Answering2020
- A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages2019
- EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks2019
- Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data2019
- Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations2018
- Fast and Scalable Expansion of Natural Language Understanding Functionality for Intelligent Agents2018
- A New Chatbot for Customer Service on Social Media2017
- Get To The Point: Summarization with Pointer-Generator Networks2017
- Length-frequency statistics for written EnglishInformation and Control, 1958