Sequence-to-Sequence Networks Learn the Meaning of Reflexive Anaphora
Preprint
Abstract
Reflexive anaphora present a challenge for semantic interpretation: their meaning varies depending on context in a way that appears to require abstract variables. Past work has raised doubts about the ability of recurrent networks to meet this challenge. In this paper, we explore this question in the context of a fragment of English that incorporates the relevant sort of contextual variability. We consider sequence-to-sequence architectures with recurrent units and show that such networks are capable of learning semantic interpretations for reflexive anaphora which generalize to novel antecedents. We explore the effect of attention mechanisms and different recurrent unit types on the type of training data that is needed for success as measured in two ways: how much lexical support is needed to induce an abstract reflexive meaning (i.e., how many distinct reflexive antecedents must occur during training) and what contexts must a noun phrase occur in to support generalization of reflexive interpretation to this noun phrase?Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence NetworksTransactions of the Association for Computational Linguistics, 2020
- Assessing the Ability of LSTMs to Learn Syntax-Sensitive DependenciesTransactions of the Association for Computational Linguistics, 2016
- Language to Logical Form with Neural Attention2016
- Effective Approaches to Attention-based Neural Machine Translation2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- The Acquisition of Anaphora by Simple Recurrent NetworksLanguage Acquisition, 2013
- Short-term memory for serial order: A recurrent neural network model.Psychological Review, 2006
- Framewise phoneme classification with bidirectional LSTM and other neural network architecturesNeural Networks, 2005
- Connectionism and cognitive architecture: A critical analysisCognition, 1988
- Proceedings of the 24th Northwest Linguistics Conference