Unsupervised Speaker Adaptation Using Attention-Based Speaker Memory for End-to-End ASR
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7384-7388
Abstract
We propose an unsupervised speaker adaptation method inspired by the neural Turing machine for end-to-end (E2E) automatic speech recognition (ASR). The proposed model contains a memory block that holds speaker i-vectors extracted from the training data and reads relevant i-vectors from the memory through an attention mechanism. The resulting memory vector (M-vector) is concatenated to the acoustic features or to the hidden layer activations of an E2E neural network model. The E2E ASR system is based on the joint connectionist temporal classification and attention-based encoder-decoder architecture. M-vector and i-vector results are compared for inserting them at different layers of the encoder neural network using the WSJ and TED-LIUM2 ASR benchmarks. We show that M-vectors, which do not require an auxiliary speaker embedding extraction system at test time, achieve similar word error rates (WERs) compared to i-vectors for single speaker utterances and significantly lower WERs for utterances in which there are speaker changes.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Hybrid CTC/Attention Architecture for End-to-End Speech RecognitionIEEE Journal of Selected Topics in Signal Processing, 2017
- Direct Acoustics-to-Word Models for English Conversational Speech Recognition2017
- Embedding-Based Speaker Adaptive Training of Deep Neural Networks2017
- I-Vectors and Structured Neural Networks for Rapid Adaptation of Acoustic ModelsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Compact Feedforward Sequential Memory Networks for Large Vocabulary Continuous Speech Recognition2016
- Context adaptive deep neural networks for fast acoustic model adaptation2015
- Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models2014
- Deep neural networks for small footprint text-dependent speaker verification2014
- Speaker adaptation of neural network acoustic models using i-vectors2013
- Maximum likelihood linear transformations for HMM-based speech recognitionComputer Speech & Language, 1998