Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection
Preprint
Abstract
Semi-supervised dialogue summarization (SSDS) leverages model-generated summaries to reduce reliance on human-labeled data and improve the performance of summarization models. While addressing label noise, previous works on semi-supervised learning primarily focus on natural language understanding tasks, assuming each sample has a unique label. However, these methods are not directly applicable to SSDS, as it is a generative task, and each dialogue can be summarized in different ways. In this work, we propose a novel scoring approach, SiCF, which encapsulates three primary dimensions of summarization model quality: Semantic invariance (indicative of model confidence), Coverage (factual recall), and Faithfulness (factual precision). Using the SiCF score, we select unlabeled dialogues with high-quality generated summaries to train summarization models. Comprehensive experiments on three public datasets demonstrate the effectiveness of SiCF scores in uncertainty estimation and semi-supervised learning for dialogue summarization tasks. Our code is available at \url{https://github.com/amazon-science/summarization-sicf-score}.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Bayesian active summarizationComputer Speech & Language, 2023
- A survey of uncertainty in deep neural networksArtificial Intelligence Review, 2023
- Self-supervised Pre-training and Semi-supervised Learning for Extractive Dialog Summarization2023
- SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in SummarizationTransactions of the Association for Computational Linguistics, 2022
- Towards a Unified Multi-Dimensional Evaluator for Text Generation2022
- Controllable Neural Dialogue Summarization with Personal Named Entity Planning2021
- DocNLI: A Large-scale Dataset for Document-level Natural Language Inference2021
- DialogSum: A Real-Life Scenario Dialogue Summarization Dataset2021
- Noisy Self-Knowledge Distillation for Text Summarization2021
- Evaluating the Factual Consistency of Abstractive Text Summarization2020