Rouge Metric Evaluation for Text Summarization Techniques
Authors:
- Marcello BarbellaMBMarcello Barbella
- Genoveffa TortoraGTGenoveffa Tortora
Preprint
- 28 May 2022
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Approaches to Automatic Text Summarization try to extract key information from one or more input texts and generate summaries whilst preserving content meaning. These strategies are separated into two groups: Extractive and Abstractive, which differ in terms of how they work. The former extracts sentences from the document text directly, whereas the latter creates a summary by interpreting the text and rewriting sentences often with new words. So, it is important to assess and confirm how similar a summary is to the original text. The question is: how can the quality of these methodologies of summaries be evaluated? For the evaluation of text summarization results, various metrics and scores have been proposed in the literature, but the most used is ROUGE.Then, the primary goal of this study is to accurately estimate the ROUGE metric's behavior. We conducted a first experiment to compare the metric efficiency for evaluating Abstractive versus Extractive Text Summarization algorithms, and a second one to compare the obtained score for two different summary approaches: a simple execution of a summarization algorithm versus a multiple execution of different algorithms on the same text. Our results have shown that: ROUGE does not obtain impressive results since it behaves in a similar way both on the Abstractive and Extractive algorithms; furthermore, in most cases, a multiple execution is better than a single one.Keywords
This publication has 5 references indexed in Scilit:
- Fuzzy logic based multi document summarization with improved sentence scoring and redundancy removal techniqueExpert Systems with Applications, 2019
- ELTS: A Brief Review for Extractive Learning-Based Text Summarizatoin Algorithms2019
- Abstractive Text Summarization based on Improved Semantic Graph ApproachInternational Journal of Parallel Programming, 2018
- An overview on Restricted Boltzmann MachinesNeurocomputing, 2018
- Text summarization using unsupervised deep learningExpert Systems with Applications, 2017