Gated-Attention Readers for Text Comprehension
Preprint
Abstract
In this paper we study the problem of answering cloze-style questions over documents. Our model, the Gated-Attention (GA) Reader, integrates a multi-hop architecture with a novel attention mechanism, which is based on multiplicative interactions between the query embedding and the intermediate states of a recurrent neural network document reader. This enables the reader to build query-specific representations of tokens in the document for accurate answer selection. The GA Reader obtains state-of-the-art results on three benchmarks for this task--the CNN \& Daily Mail news stories and the Who Did What dataset. The effectiveness of multiplicative interaction is demonstrated by an ablation study, and by comparing to alternative compositional operators for implementing the gated-attention.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Neural Semantic Encoders2017
- Broad Context Language Modeling as Reading Comprehension2017
- Attention-over-Attention Neural Networks for Reading Comprehension2017
- Natural Language Comprehension with the EpiReader2016
- Who did What: A Large-Scale Person-Centered Cloze Dataset2016
- Text Understanding with the Attention Sum Reader Network2016
- Tweet2Vec: Character-Based Distributed Representations for Social Media2016
- A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task2016
- Dynamic Entity Representation with Max-pooling Improves Machine Reading2016
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014