Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents
Authors:
- Yusen ZhangYZYusen Zhang
- Ansong NiANAnsong Ni
- Chenguang ZhuCZChenguang Zhu
- Budhaditya DebBDBudhaditya Deb
- Ahmed H. AwadallahAAAhmed H. Awadallah
- Dragomir RadevDRDragomir Radev
- Rui ZhangRZRui Zhang
Preprint
Abstract
Text summarization is an essential task to help readers capture salient information from documents, news, interviews, and meetings. However, most state-of-the-art pretrained language models are unable to efficiently process long text commonly seen in the summarization problem domain. In this paper, we propose Summ^N, a simple, flexible, and effective multi-stage framework for input texts that are longer than the maximum context lengths of typical pretrained LMs. Summ^N first generates the coarse summary in multiple stages and then produces the final fine-grained summary based on them. The framework can process input text of arbitrary length by adjusting the number of stages while keeping the LM context size fixed. Moreover, it can deal with both documents and dialogues and can be used on top of any underlying backbone abstractive summarization model. Our experiments demonstrate that Summ^N significantly outperforms previous state-of-the-art methods by improving ROUGE scores on three long meeting summarization datasets AMI, ICSI, and QMSum, two long TV series datasets from SummScreen, and a newly proposed long document summarization dataset GovReport. Our data and code are available at https://github.com/chatc/Summ-N.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- An Exploratory Study on Long Dialogue Summarization: What Works and What’s Next2021
- ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining2021
- Efficient Attentions for Long Document Summarization2021
- Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs2021
- QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization2021
- A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining2020
- On Extractive and Abstractive Neural Document Summarization with Transformer Language Models2020
- Coarse-to-Fine Query Focused Multi-Document Summarization2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts2018