Chain-Structure Echo State Network With Stochastic Optimization: Methodology and Application
- 29 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 33 (5) , 1974-1985
Abstract
In this article, a chain-structure echo state network (CESN) with stacked subnetwork modules is newly proposed as a new kind of deep recurrent neural network for multivariate time series prediction. Motivated by the philosophy of “divide and conquer,” the related input vectors are first divided into clusters, and the final output results of CESN are then integrated by successively learning the predicted values of each clustered variable. Network structure, mathematical model, training mechanism, and stability analysis are, respectively, studied for the proposed CESN. In the training stage, least-squares regression is first used to pretrain the output weights in a module-by-module way, and stochastic local search (SLS) is developed to fine-tune network weights toward global optima. The loss function of CESN can be effectively reduced by SLS. To avoid overfitting, the optimization process is stopped when the validation error starts to increase. Finally, SLS-CESN is evaluated in chaos prediction benchmarks and real applications. Four different examples are given to verify the effectiveness and robustness of CESN and SLS-CESN.Keywords
Funding Information
- National Natural Science Foundation of China (61803054, 61972112, U20A2031)
This publication has 31 references indexed in Scilit:
- Two Machine Learning Approaches for Short-Term Wind Speed Time-Series PredictionIEEE Transactions on Neural Networks and Learning Systems, 2015
- Spatial-temporal forecasting of solar radiationRenewable Energy, 2015
- Echo State Neural Network Based State Feedback Control for SISO Afine Nonlinear SystemsIFAC-PapersOnLine, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Adaptively weighted support vector regression for financial time series prediction2014
- A new cascade NN based method to short-term load forecast in deregulated electricity marketEnergy Conversion and Management, 2013
- A Practical Guide to Applying Echo State Networks2012
- Stable Output Feedback in Reservoir Computing Using Ridge Regression2008
- Optimization and applications of echo state networks with leaky- integrator neuronsNeural Networks, 2007
- Solar Irradiance2004