Optimizing neural network hyperparameters with Gaussian processes for dialog act classification
Authors:
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 406-413
Abstract
Systems based on artificial neural networks (ANNs) have achieved state-of-the-art results in many natural language processing tasks. Although ANNs do not require manually engineered features, ANNs have many hyperparameters to be optimized. The choice of hyperparameters significantly impacts models' performances. However, the ANN hyperparameters are typically chosen by manual, grid, or random search, which either requires expert experiences or is computationally expensive. Recent approaches based on Bayesian optimization using Gaussian processes (GPs) is a more systematic way to automatically pinpoint optimal or near-optimal machine learning hyperparameters. Using a previously published ANN model yielding state-of-the-art results for dialog act classification, we demonstrate that optimizing hyperparameters using GP further improves the results, and reduces the computational time by a factor of 4 compared to a random search. Therefore it is a useful technique for tuning ANN models to yield the best performances for NLP tasks.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Neural Architectures for Named Entity Recognition2016
- Gaussian Process-Based Feature Selection for Wavelet Parameters: Predicting Acute Hypotensive Episodes from Physiological Signals2015
- Non-lexical neural architecture for fine-grained POS Tagging2015
- A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering2015
- Recurrent Neural Networks for Word Alignment Model2014
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- Practical Recommendations for Gradient-Based Training of Deep Architectures2012
- The ICSI Meeting Corpus2003
- Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational SpeechComputational Linguistics, 2000