Importance-Aware Data Augmentation for Document-Level Neural Machine Translation
Authors:
Preprint
Abstract
Document-level neural machine translation (DocNMT) aims to generate translations that are both coherent and cohesive, in contrast to its sentence-level counterpart. However, due to its longer input length and limited availability of training data, DocNMT often faces the challenge of data sparsity. To overcome this issue, we propose a novel Importance-Aware Data Augmentation (IADA) algorithm for DocNMT that augments the training data based on token importance information estimated by the norm of hidden states and training gradients. We conduct comprehensive experiments on three widely-used DocNMT benchmarks. Our empirical results show that our proposed IADA outperforms strong DocNMT baselines as well as several data augmentation approaches, with statistical significance on both sentence-level and document-level BLEU.Keywords
All Related Versions
This publication has 42 references indexed in Scilit:
- Deep multi-view learning methods: A reviewNeurocomputing, 2021
- Text Data Augmentation for Deep LearningJournal of Big Data, 2021
- Measuring and Increasing Context Usage in Context-Aware Machine Translation2021
- G-Transformer for Document-Level Machine Translation2021
- Multilingual Denoising Pre-training for Neural Machine TranslationTransactions of the Association for Computational Linguistics, 2020
- Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics2020
- Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns2020
- BPE-Dropout: Simple and Effective Subword Regularization2020
- A Simple and Effective Unified Encoder for Document-Level Machine Translation2020
- The Automatic Creation of Literature AbstractsIBM Journal of Research and Development, 1958