Korean-Vietnamese Neural Machine Translation System With Korean Morphological Analysis and Word Sense Disambiguation
Authors:
Open Access
- 5 March 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 32602-32616
Abstract
Although deep neural networks have recently led to great achievements in machine translation (MT), various challenges are still encountered during the development of Korean-Vietnamese MT systems. Because Korean is a morphologically rich language and Vietnamese is an analytic language, neither have clear word boundaries. The high rate of homographs in Korean causes word ambiguities, which causes problems in neural MT (NMT). In addition, as a low-resource language pair, there is no freely available, adequate Korean-Vietnamese parallel corpus that can be used to train translation models. In this paper, we manually established a lexical semantic network for the special characteristics of Korean as a knowledge base that was used for developing our Korean morphological analysis and word-sense disambiguation system: UTagger. We also constructed a large Korean-Vietnamese parallel corpus, in which we applied the state-of-the-art Vietnamese word segmentation method RDRsegmenter to Vietnamese texts and UTagger to Korean texts. Finally, we built a bi-directional Korean-Vietnamese NMT system based on the attention-based encoder-decoder architecture. The experimental results indicated that UTagger and RDRsegmenter could significantly improve the performance of the Korean-Vietnamese NMT system, achieving remarkable results by 27.79 BLEU points and 58.77 TER points in Korean-to-Vietnamese direction and 25.44 BLEU points and 58.72 TER points in the reverse direction.Keywords
Funding Information
- National Research Foundation of Korea
- Korean government (NRF-2017M3C4A7068187)
- ICT R&D program of MSIP/IITP
This publication has 31 references indexed in Scilit:
- Conditional Random Fields for Korean Morpheme Segmentation and POS TaggingACM Transactions on Asian and Low-Resource Language Information Processing, 2015
- Addressing the Rare Word Problem in Neural Machine Translation2015
- On Using Very Large Target Vocabulary for Neural Machine Translation2015
- Korean Homograph Tagging Model based on Sub-Word Conditional ProbabilityKIPS Transactions on Software and Data Engineering, 2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- A Hybrid Approach to Word Segmentation of Vietnamese TextsLecture Notes in Computer Science, 2008
- Syllable-Pattern-Based Unknown-Morpheme Segmentation and Estimation for Hybrid Part-of-Speech Tagging of KoreanComputational Linguistics, 2002
- Long Short-Term MemoryNeural Computation, 1997
- WordNetCommunications of the ACM, 1995
- Syllable-based model for the Korean morphology1994