Catch-22s of reservoir computing
Open Access
- 25 September 2023
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review Research
- Vol. 5 (3) , 033213
Abstract
Reservoir computing (RC) is a simple and efficient model-free framework for forecasting the behavior of nonlinear dynamical systems from data. Here, we show that there exist commonly-studied systems for which leading RC frameworks struggle to learn the dynamics unless key information about the underlying system is already known. We focus on the important problem of basin prediction—determining which attractor a system will converge to from its initial conditions. First, we show that the predictions of standard RC models (echo state networks) depend critically on warm-up time, requiring a warm-up trajectory containing almost the entire transient in order to identify the correct attractor. Accordingly, we turn to next-generation reservoir computing (NGRC), an attractive variant of RC that requires negligible warm-up time. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction, even with sparse training data (e.g., a single transient trajectory). Yet, a tiny uncertainty in the exact nonlinearity can render prediction accuracy no better than chance. Our results highlight the challenges faced by data-driven methods in learning the dynamics of multistable systems and suggest potential avenues to make these approaches more robust.Keywords
All Related Versions
Funding Information
- Natural Sciences and Engineering Research Council of Canada (RGPIN-2020-05015)
This publication has 66 references indexed in Scilit:
- Discovering governing equations from data by sparse identification of nonlinear dynamical systemsProceedings of the National Academy of Sciences, 2016
- How dead ends undermine power grid stabilityNature Communications, 2014
- Doubly Transient Chaos: Generic Form of Chaos in Autonomous Dissipative SystemsPhysical Review Letters, 2013
- How basin stability complements the linear-stability paradigmNature Physics, 2013
- Information processing using a single dynamical node as complex systemNature Communications, 2011
- Reservoir computing approaches to recurrent neural network trainingComputer Science Review, 2009
- The size of the sync basinChaos: An Interdisciplinary Journal of Nonlinear Science, 2006
- Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless CommunicationScience, 2004
- Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on PerturbationsNeural Computation, 2002
- Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, 1982