Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement
Authors:
Preprint
Abstract
We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine translation (En-De and En-Ro) and image caption generation, and observe that it significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- State-of-the-Art Speech Recognition with Sequence-to-Sequence Models2018
- Towards Decoding as Continuous Optimisation in Neural Machine Translation2017
- Deep Residual Learning for Image Recognition2016
- Neural Machine Translation of Rare Words with Subword Units2016
- Learning Distributed Representations of Sentences from Unlabelled Data2016
- Describing Multimedia Content Using Attention-Based Encoder-Decoder NetworksIEEE Transactions on Multimedia, 2015
- Recurrent Continuous Translation Models2013