Deep learning-based modified transformer model for automated news article summarization
Open Access
- 1 January 2024
- journal article
- research article
- Published by National Library of Serbia in Facta universitatis - series: Electronics and Energetics
- Vol. 37 (2) , 261-276
Abstract
The amount of textual data on the internet is increasing enormously, so data summarization into text has become essential. As generating text summaries manually is an arduous task and humans are generally prone to make mistakes, deep learning techniques have evolved to overcome this problem. Modified transformer-based deep learning models with varying encoder-decoder and feed-forward network layers are proposed to develop an abstractive summary of the news articles. The proposed transformer model provides the advantage of parallelization with the help of multiple attention head layers to process long sentences, and hence, better text summarization performance is achieved. These models are trained on an ?in-shorts? dataset, and the proposed model is compared with the PEGASUS-CNNdaily-mail, BART-large-CNN, and DistilBART-CNN-12-6 models on the CNN/DailyMail dataset. The performance is evaluated in terms of the ROUGE score by comparing it with the existing Recurrent Neural Network (RNN) model. The suggested transformer model achieved a ROUGE score of 0.33, surpassing the RNN model score of 0.17. This innovative approach can be employed on extensive textual data to extract summaries or headlines.Keywords
This publication has 17 references indexed in Scilit:
- Automatic text summarization: A comprehensive surveyExpert Systems with Applications, 2021
- Recurrent Neural Network based Text Summarization Techniques by Word Sequence Generation2021
- Neural Abstractive Text Summarization with Sequence-to-Sequence ModelsACM/IMS Transactions on Data Science, 2021
- Automatic Text Summarization Using Transformers2021
- A Comprehensive Survey on Transfer LearningProceedings of the IEEE, 2021
- Deep Learning Based Abstractive Text Summarization: Approaches, Datasets, Evaluation Measures, and ChallengesMathematical Problems in Engineering, 2020
- Segmentation of Multi-Modal MRI Brain Tumor Sub-Regions Using Deep LearningJournal of Electrical Engineering & Technology, 2020
- Abstractive summarization: An overview of the state of the artExpert Systems with Applications, 2019
- Multi-Task Learning for Abstractive and Extractive SummarizationData Science and Engineering, 2019
- Abstractive Text Summarization based on Improved Semantic Graph ApproachInternational Journal of Parallel Programming, 2018