MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes
Preprint
Abstract
We introduce MemSum (Multi-step Episodic Markov decision process extractive SUMmarizer), a reinforcement-learning-based extractive summarizer enriched at each step with information on the current extraction history. When MemSum iteratively selects sentences into the summary, it considers a broad information set that would intuitively also be used by humans in this task: 1) the text content of the sentence, 2) the global text context of the rest of the document, and 3) the extraction history consisting of the set of sentences that have already been extracted. With a lightweight architecture, MemSum obtains state-of-the-art test-set performance (ROUGE) in summarizing long documents taken from PubMed, arXiv, and GovReport. Ablation studies demonstrate the importance of local, global, and history information. A human evaluation confirms the high quality and low redundancy of the generated summaries, stemming from MemSum's awareness of extraction history.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Efficient Attentions for Long Document Summarization2021
- A hierarchical self-attentive neural extractive summarizer via reinforcement learning (HSASRL)Applied Intelligence, 2020
- On Extractive and Abstractive Neural Document Summarization with Transformer Language Models2020
- A Divide-and-Conquer Approach to the Summarization of Long DocumentsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Recent automatic text summarization techniques: a surveyArtificial Intelligence Review, 2016
- A Survey of Text Summarization Techniques2012
- Wilcoxon Signed‐Rank Test2008
- Long Short-Term MemoryNeural Computation, 1997
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992