AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference
Authors:
- Yang HanYHYang Han
- Rui WangRWRui Wang
- Lu ChenLCLu Chen
- Kai YuKai Yu
Preprint
Abstract
Text summarization tasks commonly employ Pre-trained Language Models (PLMs) to fit diverse standard datasets. While these PLMs excel in automatic evaluations, they frequently underperform in human evaluations, indicating a deviation between their generated summaries and human summarization preferences. This discrepancy is likely due to the low quality of fine-tuning datasets and the limited availability of high-quality human-annotated data that reflect true human preference. To address this challenge, we introduce a novel human summarization preference alignment framework AlignSum. This framework consists of three parts: Firstly, we construct a Data Pymarid with extractive, abstractive, and human-annotated summary data. Secondly, we conduct the Gaussian Resampling to remove summaries with extreme lengths. Finally, we implement the two-stage hierarchical fine-tuning with Data Pymarid after Gaussian Resampling. We apply AlignSum to PLMs on the human-annotated CNN/DailyMail and BBC XSum datasets. Experiments show that with AlignSum, PLMs like BART-Large surpass 175B GPT-3 in both automatic and human evaluations. This demonstrates that AlignSum significantly enhances the alignment of language models with human summarization preferences.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization2024
- MACSum: Controllable Summarization with Mixed AttributesTransactions of the Association for Computational Linguistics, 2023
- BRIO: Bringing Order to Abstractive Summarization2022
- PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization2022
- AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization2021
- A Joint Sentence Scoring and Selection Framework for Neural Extractive Document SummarizationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks2020
- How many words do we read per minute? A review and meta-analysis of reading rateJournal of Memory and Language, 2019
- Text Summarization Techniques: A Brief SurveyInternational Journal of Advanced Computer Science and Applications, 2017