Domain Transfer for Deep Natural Language Generation from Abstract Meaning Representations
Author:
- 18 July 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computational Intelligence Magazine
- Vol. 12 (3) , 18-28
Abstract
Stochastic natural language generation systems that are trained from labelled datasets are often domain-specific in their annotation and in their mapping from semantic input representations to lexical-syntactic outputs. As a result, learnt models fail to generalize across domains, heavily restricting their usability beyond single applications. In this article, we focus on the problem of domain adaptation for natural language generation. We show how linguistic knowledge from a source domain, for which labelled data is available, can be adapted to a target domain by reusing training data across domains. As a key to this, we propose to employ abstract meaning representations as a common semantic representation across domains. We model natural language generation as a long short-term memory recurrent neural network encoder-decoder, in which one recurrent neural network learns a latent representation of a semantic input, and a second recurrent neural network learns to decode it to a sequence of words. We show that the learnt representations can be transferred across domains and can be leveraged effectively to improve training on new unseen domains. Experiments in three different domains and with six datasets demonstrate that the lexical-syntactic constructions learnt in one domain can be transferred to new domains and achieve up to 75-100% of the performance of in-domain training. This is based on objective metrics such as BLEU and semantic error rate and a subjective human rating study. Training a policy from prior knowledge from a different domain is consistently better than pure in-domain training by up to 10%.Keywords
This publication has 39 references indexed in Scilit:
- Toward Abstractive Summarization Using Semantic Representations2015
- Hierarchical reinforcement learning for situated natural language generationNatural Language Engineering, 2014
- How to make words with vectors: Phrase generation in distributional semantics2014
- A Discriminative Graph-Based Parser for the Abstract Meaning Representation2014
- ReferItGame: Referring to Objects in Photographs of Natural Scenes2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Adaptation of maximum entropy capitalizer: Little data can help a lotComputer Speech & Language, 2006
- Long Short-Term MemoryNeural Computation, 1997
- Enabling technology for multilingual natural language generation: the KPML development environmentNatural Language Engineering, 1997
- Meaning-Text Models: A Recent Trend in Soviet LinguisticsAnnual Review of Anthropology, 1981