AutoML With Parallel Genetic Algorithm for Fast Hyperparameters Optimization in Efficient IoT Time Series Prediction
- 22 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 19 (9) , 9555-9564
Abstract
With the development of artificial intelligence and the improvement of hardware computing power, deep learning models have become widely used in the Internet of Things (IoT) field, especially for analyzing spatiotemporal data collected by wireless sensors. Recurrent Neural Networks (RNN) such as Long Short-Term Memory (LSTM) network are generally used for these time series data. Hyperparameter settings of model training are regarded as essential factors for the performance of deep learning models. Manually optimizing hyperparameters not only cost more resources but also be more likely to set hyperparameters that follow stereotypes, resulting in unreasonable hyperparameter settings and poor model performance. As one of the most important fields in Automated Machine Learning (AutoML) research, automated hyper-parameter optimization (HPO) mainly includes grid search, hyperparameter search based on genetic algorithm, etc. Whereas these methods have their own drawbacks. In this paper, an automated hyper-parameter optimization method based on parallel genetic algorithm (PGA) is proposed. According to the process of PGA, this paper divided hyperparameter optimization into several stages, including population initialization, fitness function, tournament selection, crossover operators, mutation operators, subgroup exchange, and end of evolution. Then the proposed hyperparameter optimization method is implemented in LSTM models and tested on two different time series datasets collected by real-world IoT sensors. By comparing our proposed method with other mainstream HPO methods in different datasets, it is proved that our HPO method based on PGA shows better performance on both time costs and prediction results.Keywords
Funding Information
- National Natural Science Foundation of China (61972145)
- National Key R&D Program of China (2019YFB1405703)
This publication has 22 references indexed in Scilit:
- Hyperparameter Optimization in Convolutional Neural Network using Genetic AlgorithmsInternational Journal of Advanced Computer Science and Applications, 2019
- Highly Accurate Machine Fault Diagnosis Using Deep Transfer LearningIEEE Transactions on Industrial Informatics, 2018
- Machine learning for internet of things data analysis: a surveyDigital Communications and Networks, 2018
- AutoLearn — Automated Feature Generation and Selection2017
- A Parallel Genetic Algorithm Based Feature Selection and Parameter Optimization for Support Vector MachineScientific Programming, 2016
- Optimizing deep learning hyper-parameters through an evolutionary algorithm2015
- A Survey of System Architecture Requirements for Health Care-Based Wireless Sensor NetworksSensors, 2011
- Long Short-Term MemoryNeural Computation, 1997
- Learning representations by back-propagating errorsNature, 1986
- Fitting Segmented Regression Models by Grid SearchJournal of the Royal Statistical Society Series C: Applied Statistics, 1980