Beyond Offline Mapping: Learning Cross Lingual Word Embeddings through Context Anchoring
Authors:
Preprint
Abstract
Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embeddings. Such methods critically rely on those embeddings having a similar structure, but it was recently shown that the separate training in different languages causes departures from this assumption. In this paper, we propose an alternative approach that does not have this limitation, while requiring a weak seed dictionary (e.g., a list of identical words) as the only form of supervision. Rather than aligning two fixed embedding spaces, our method works by fixing the target language embeddings, and learning a new set of embeddings for the source language that are aligned with them. To that end, we use an extension of skip-gram that leverages translated context words as anchor points, and incorporates self-learning and iterative restarts to reduce the dependency on the initial dictionary. Our approach outperforms conventional mapping methods on bilingual lexicon induction, and obtains competitive results in the downstream XNLI task.Keywords
All Related Versions
- Version 1, 2020-12-31, ArXiv
- Version 2, 2021-08-03, ArXiv
- Published version: , 6479.
This publication has 18 references indexed in Scilit:
- LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space2020
- Are All Good Word Vector Spaces Isomorphic?2020
- Non-Linear Instance-Based Cross-Lingual Mapping for Non-Isomorphic Embedding Spaces2020
- On the Cross-lingual Transferability of Monolingual Representations2020
- Learning Unsupervised Word Mapping via Maximum Mean Discrepancy2019
- Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction2019
- XNLI: Evaluating Cross-lingual Sentence Representations2018
- NORMA: Neighborhood Sensitive Maps for Multilingual Word Embeddings2018
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- Bilingual Distributed Word Representations from Document-Aligned Comparable DataJournal of Artificial Intelligence Research, 2016