SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations
Preprint
Abstract
End-to-end Speech Translation is hindered by a lack of available data resources. While most of them are based on documents, a sentence-level version is available, which is however single and static, potentially impeding the usefulness of the data. We propose a new data augmentation strategy, SegAugment, to address this issue by generating multiple alternative sentence-level versions of a dataset. Our method utilizes an Audio Segmentation system, which re-segments the speech of each document with different length constraints, after which we obtain the target text via alignment methods. Experiments demonstrate consistent gains across eight language pairs in MuST-C, with an average increase of 2.5 BLEU points, and up to 5 BLEU for low-resource scenarios in mTEDx. Furthermore, when combined with a strong system, SegAugment establishes new state-of-the-art results in MuST-C. Finally, we show that the proposed method can also successfully augment sentence-level datasets, and that it enables Speech Translation models to close the gap between the manual and automatic segmentation at inference time.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Data Augmenting Contrastive Learning of Speech Representations in the Time Domain2021
- On The Usefulness of Self-Attention for Automatic Speech Recognition with Transformers2021
- Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation2020
- SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation2020
- Speech Translation and the End-to-End Promise: Taking Stock of Where We Are2020
- SRPOL’s System for the IWSLT 2020 End-to-End Speech Translation Task2020
- End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT20202020
- Leveraging Weakly Supervised Data to Improve End-to-end Speech-to-text Translation2019
- Librispeech: An ASR corpus based on public domain audio books2015
- Connectionist temporal classification2006