Multilingual AMR-to-Text Generation
Preprint
Abstract
Generating text from structured data is challenging because it requires bridging the gap between (i) structure and natural language (NL) and (ii) semantically underspecified input and fully specified NL output. Multilingual generation brings in an additional challenge: that of generating into languages with varied word order and morphological properties. In this work, we focus on Abstract Meaning Representations (AMRs) as structured input, where previous research has overwhelmingly focused on generating only into English. We leverage advances in cross-lingual embeddings, pretraining, and multilingual models to create multilingual AMR-to-text models that generate in twenty one different languages. For eighteen languages, based on automatic metrics, our multilingual models surpass baselines that generate into a single language. We analyse the ability of our multilingual models to accurately capture morphology and word order using human evaluation, and find that native speakers judge our generations to be fluent.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web2021
- Leveraging Pre-trained Checkpoints for Sequence Generation TasksTransactions of the Association for Computational Linguistics, 2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Unsupervised Cross-lingual Representation Learning at Scale2020
- MLQA: Evaluating Cross-lingual Extractive Question Answering2020
- Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation2020
- Attending to Future Tokens for Bidirectional Sequence Generation2019
- Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT2019
- Multi-way, multilingual neural machine translationComputer Speech & Language, 2017
- Neural Text Generation from Structured Data with Application to the Biography Domain2016