Unified extractive-abstractive summarization: a hybrid approach utilizing BERT and transformer models for enhanced document summarization
Open Access
- 18 November 2024
- journal article
- research article
- Published by PeerJ in PeerJ Computer Science
- Vol. 10, e2424
Abstract
With the exponential proliferation of digital documents, there arises a pressing need for automated document summarization (ADS). Summarization, a compression technique, condenses a source document into concise sentences that encapsulate its salient information for summary generation. A primary challenge lies in crafting a dependable summary, contingent upon both extracted features and human-established parameters. This article introduces an intelligent methodology that seamlessly integrates extractive and abstractive techniques to ensure heightened relevance between the input document and its summary. Initially, input sentences undergo transformation into representations utilizing BERT, subsequently transposed into a symmetric matrix based on their similarity. Semantically congruent sentences are then extracted from this matrix to construct an extractive summary. The transformer model integrates an objective function highly symmetric and invariant under unitary transformation for language generation. This model refines the extracted informative sentences and generates an abstractive summary akin to manually crafted summaries. Employing this hybrid summarization technique on the CNN/DailyMail dataset and DUC2004, we evaluate its efficacy using ROUGE metrics. Results demonstrate the superiority of our proposed technique over conventional summarization methods.Keywords
This publication has 50 references indexed in Scilit:
- Leveraging Pre-trained Checkpoints for Sequence Generation TasksTransactions of the Association for Computational Linguistics, 2020
- Text Summarization Method Based on Double Attention Pointer NetworkIEEE Access, 2020
- Language Model-Driven Topic Clustering and Summarization for News ArticlesIEEE Access, 2019
- Exploring Human-Like Reading Strategy for Abstractive Text SummarizationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Learning Relevant Models using Symbolic Regression for Automatic Text SummarizationComputación y Sistemas, 2019
- Learning Latent Factors for Community Identification and SummarizationIEEE Access, 2018
- A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)IEEE Access, 2018
- Calculating the Upper Bounds for Multi-Document Summarization using Genetic AlgorithmsComputación y Sistemas, 2018
- SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of DocumentsProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Recent automatic text summarization techniques: a surveyArtificial Intelligence Review, 2016