A training algorithm for networks of high-variability reservoirs
Open Access
- 2 September 2020
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 10 (1) , 1-11
Abstract
Physical reservoir computing approaches have gained increased attention in recent years due to their potential for low-energy high-performance computing. Despite recent successes, there are bounds to what one can achieve simply by making physical reservoirs larger. Therefore, we argue that a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that error backpropagation cannot be used directly to train a large class of multi-reservoir systems, we propose an alternative framework that combines the power of backpropagation with the speed and simplicity of classic training algorithms. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach. We train a network of 3 Echo State Networks to perform the well-known NARMA-10 task, where we use intermediate targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in an efficient way.Keywords
This publication has 41 references indexed in Scilit:
- Experimental demonstration of reservoir computing on a silicon photonics chipNature Communications, 2014
- Nanophotonic Reservoir Computing With Photonic Crystal Cavities to Generate Periodic PatternsIEEE Transactions on Neural Networks and Learning Systems, 2013
- Representation Learning: A Review and New PerspectivesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- An experimental unification of reservoir computing methodsNeural Networks, 2007
- New results on recurrent network training: unifying the algorithms and accelerating convergenceIEEE Transactions on Neural Networks, 2000
- Learning representations by back-propagating errorsNature, 1986
- Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approachJournal of Mathematical Physics, 1981
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970
- The numerical solution of variational problemsJournal of Mathematical Analysis and Applications, 1962
- Gradient Theory of Optimal Flight PathsARS Journal, 1960