Improving Statistical Machine Translation Using Bayesian Word Alignment and Gibbs Sampling
Authors:
- 1 February 2013
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech, and Language Processing
- Vol. 21 (5) , 1090-1101
Abstract
We present a Bayesian approach to word alignment inference in IBM Models 1 and 2. In the original approach, word translation probabilities (i.e., model parameters) are estimated using the expectation-maximization (EM) algorithm. In the proposed approach, they are random variables with a prior and are integrated out during inference. We use Gibbs sampling to infer the word alignment posteriors. The inferred word alignments are compared against EM and variational Bayes (VB) inference in terms of their end-to-end translation performance on several language pairs and types of corpora up to 15 million sentence pairs. We show that Bayesian inference outperforms both EM and VB in the majority of test cases. Further analysis reveals that the proposed method effectively addresses the high-fertility rare word problem in EM and unaligned rare word problem in VB, achieves higher agreement and vocabulary coverage rates than both, and leads to smaller phrase tables.Keywords
This publication has 26 references indexed in Scilit:
- Z-MERT: A Fully Configurable Open Source Tool for Minimum Error Rate Training of Machine Translation SystemsThe Prague Bulletin of Mathematical Linguistics, 2009
- A Gibbs sampler for phrasal synchronous grammar induction2009
- Bayesian semi-supervised Chinese word segmentation for statistical machine translation2008
- Sampling alignment structure under a Bayesian translation model2008
- Moses2007
- Comparative study on corpora for speech translationIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Statistical phrase-based translation2003
- A comparison of alignment models for statistical machine translation2000
- An empirical study of smoothing techniques for language modelingComputer Speech & Language, 1999
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1984