Unsupervised Word Translation with Adversarial Autoencoder
Authors:
Open Access
- 1 June 2020
- journal article
- research article
- Published by MIT Press in Computational Linguistics
- Vol. 46 (2) , 257-288
Abstract
Crosslingual word embeddings learned from monolingual embeddings have a crucial role in many downstream tasks, ranging from machine translation to transfer learning. Adversarial training has shown impressive success in learning crosslingual embeddings and the associated word translation task without any parallel data by mapping monolingual embeddings to a shared space. However, recent work has shown superior performance for non-adversarial methods in more challenging language pairs. In this article, we investigate adversarial autoencoder for unsupervised word translation and propose two novel extensions to it that yield more stable training and improved results. Our method includes regularization terms to enforce cycle consistency and input reconstruction, and puts the target encoders as an adversary against the corresponding discriminator. We use two types of refinement procedures sequentially after obtaining the trained encoders and mappings from the adversarial training, namely, refinement with Procrustes solution and refinement with symmetric re-weighting. Extensive experimentations with high- and low-resource languages from two different data sets show that our method achieves better performance than existing adversarial and non-adversarial approaches and is also competitive with the supervised system. Along with performing comprehensive ablation studies to understand the contribution of different components of our adversarial model, we also conduct a thorough analysis of the refinement procedures to understand their effects.Keywords
This publication has 18 references indexed in Scilit:
- Phrase-Based & Neural Unsupervised Machine Translation2018
- Non-Adversarial Unsupervised Word Translation2018
- A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings2018
- Gromov-Wasserstein Alignment of Word Embedding Spaces2018
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- Learning bilingual word embeddings with (almost) no bilingual data2017
- Learning principled bilingual mappings of word embeddings while preserving monolingual invariance2016
- Towards cross-lingual distributed representations without parallel text trained with adversarial autoencoders2016
- Bilingual Word Representations with Monolingual Quality in Mind2015
- Improving Vector Space Word Representations Using Multilingual Correlation2014