New types of deep neural network learning for speech recognition and related applications: an overview
Authors:
Top Cited Papers
- 1 May 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15206149,p. 8599-8603
Abstract
In this paper, we provide an overview of the invited and contributed papers presented at the special session at ICASSP-2013, entitled “New Types of Deep Neural Network Learning for Speech Recognition and Related Applications,” as organized by the authors. We also describe the historical context in which acoustic models based on deep neural networks have been developed. The technical overview of the papers presented in our special session is organized into five ways of improving deep learning methods: (1) better optimization; (2) better types of neural activation function and better network architectures; (3) better ways to determine the myriad hyper-parameters of deep neural networks; (4) more appropriate ways to preprocess speech for deep neural networks; and (5) ways of leveraging multiple languages or dialects that are more easily achieved with deep neural networks than with Gaussian mixture models.Keywords
This publication has 29 references indexed in Scilit:
- Speech recognition with deep recurrent neural networks2013
- A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion2013
- Multilingual acoustic models using distributed deep neural networks2013
- Advances in optimizing recurrent networks2013
- Deep convolutional neural networks for LVCSR2013
- On rectified linear units for speech processing2013
- Reducing the Dimensionality of Data with Neural NetworksScience, 2006
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- A time-delay neural network architecture for isolated word recognitionNeural Networks, 1990