Long short-term memory language models with additive morphological features for automatic speech recognition
Authors:
- 1 April 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5246-5250
Abstract
Models of morphologically rich languages suffer from data sparsity when words are treated as atomic units. Word-based language models cannot transfer knowledge from common word forms to rarer variant forms. Learning a continuous vector representation of each morpheme allows a compositional model to represent a word as the sum of its constituent morphemes' vectors. Rare and unknown words containing common morphemes can thus be represented with greater fidelity despite their sparsity. Our novel neural network language model integrates this additive morphological representation into a long short-term memory architecture, improving Russian speech recognition word error rates by 0.9 absolute, 4.4% relative, compared to a robust n-gram baseline model.Keywords
This publication has 6 references indexed in Scilit:
- Morpheme-based feature-rich language models using Deep Neural Networks for LVCSR of Egyptian Arabic2013
- A unified architecture for natural language processing2008
- Continuous space language modelsComputer Speech & Language, 2007
- Efficient training of large neural networks for language modeling2005
- Long Short-Term MemoryNeural Computation, 1997
- Learning long-term dependencies with gradient descent is difficultIEEE Transactions on Neural Networks, 1994