Word alignment in the Russian-Chinese parallel corpus
- 18 October 2023
- book chapter
- Published by John Benjamins Publishing Company
- p. 195-215
Abstract
The Russian-Chinese parallel corpus (RuZhCorp) was created in 2016 by sinologists and computational linguists. So far, it has accumulated 1 074 texts and over 4.6 million words that are aligned on a sentence level. To produce word alignment for the entire corpus, we used deep neural networks trained both on the whole RuZhCorp and on a manually aligned at a word level gold dataset. Using the principles presented in previous publications, we compiled the first word-to-word alignment guideline for the Russian-Chinese language pair, which makes the manual alignment process less ambiguous and more consistent. The joint fine-tuning of the LaBSE deep learning model on RuZhCorp and the gold dataset achieved the best AER of 18.9%.Keywords
This publication has 20 references indexed in Scilit:
- Biasing Attention-Based Recurrent Neural Networks Using External Alignment Information2017
- 6. Studying colexification through massively parallell corpora2016
- Lexical typology through similarity semantics: Toward a semantic map of motion verbsLinguistics, 2012
- Parallel texts: using translational equivalents in linguistic typologySTUF - Language Typology and Universals, 2007
- Guidelines for Word Alignment Evaluation and Manual AlignmentLanguage Resources and Evaluation, 2005
- Automatic bilingual lexicon acquisition using random indexing of parallel corporaNatural Language Engineering, 2005
- A Systematic Comparison of Various Statistical Alignment ModelsComputational Linguistics, 2003
- Improved statistical alignment models2000
- Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the Web1999
- Existentials and Other LocativesLanguage, 1992