Evolving Deep Networks Using HPC
Open Access
- 12 November 2017
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Abstract
While a large number of deep learning networks have been studied and published that produce outstanding results on natural image datasets, these datasets only make up a fraction of those to which deep learning can be applied. These datasets include text data, audio data, and arrays of sensors that have very different characteristics than natural images. As these "best" networks for natural images have been largely discovered through experimentation and cannot be proven optimal on some theoretical basis, there is no reason to believe that they are the optimal network for these drastically different datasets. Hyperparameter search is thus often a very important process when applying deep learning to a new problem. In this work we present an evolutionary approach to searching the possible space of network hyperparameters and construction that can scale to 18, 000 nodes. This approach is applied to datasets of varying types and characteristics where we demonstrate the ability to rapidly find best hyperparameters in order to enable practitioners to quickly iterate between idea and result.Keywords
Funding Information
- U.S Department of Energy (DE-AC05-00OR22725)
This publication has 14 references indexed in Scilit:
- A convolutional neural network neutrino event classifierJournal of Instrumentation, 2016
- Deep learning for computational biologyMolecular Systems Biology, 2016
- Deep Residual Learning for Image Recognition2016
- Optimizing deep learning hyper-parameters through an evolutionary algorithm2015
- Going deeper with convolutions2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Caffe2014
- The data acquisition system and infrastructureNuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 2012
- Practical Recommendations for Gradient-Based Training of Deep Architectures2012
- Evolving Neural Networks through Augmenting TopologiesEvolutionary Computation, 2002