Catch-22s of reservoir computing
Authors:
Preprint
Abstract
Reservoir Computing (RC) is a simple and efficient model-free framework for forecasting the behavior of nonlinear dynamical systems from data. Here, we show that there exist commonly-studied systems for which leading RC frameworks struggle to learn the dynamics unless key information about the underlying system is already known. We focus on the important problem of basin prediction -- determining which attractor a system will converge to from its initial conditions. First, we show that the predictions of standard RC models (echo state networks) depend critically on warm-up time, requiring a warm-up trajectory containing almost the entire transient in order to identify the correct attractor. Accordingly, we turn to Next-Generation Reservoir Computing (NGRC), an attractive variant of RC that requires negligible warm-up time. By incorporating the exact nonlinearities in the original equations, we show that NGRC can accurately reconstruct intricate and high-dimensional basins of attraction, even with sparse training data (e.g., a single transient trajectory). Yet, a tiny uncertainty in the exact nonlinearity can render prediction accuracy no better than chance. Our results highlight the challenges faced by data-driven methods in learning the dynamics of multistable systems and suggest potential avenues to make these approaches more robust.Keywords
All Related Versions
This publication has 60 references indexed in Scilit:
- Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative studyMachine Learning with Applications, 2022
- A Hybrid Approach to Atmospheric Modeling That Combines Machine Learning With a Physics‐Based Numerical ModelJournal of Advances in Modeling Earth Systems, 2022
- Statistically derived geometrical landscapes capture principles of decision-making dynamics during cell fate transitionsCell Systems, 2022
- Echo State Networks trained by Tikhonov least squares are approximators of ergodic dynamical systemsPhysica D: Nonlinear Phenomena, 2021
- Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationActa Numerica, 2021
- Backpropagation algorithms and Reservoir Computing in Recurrent Neural Networks for the forecasting of complex spatiotemporal dynamicsNeural Networks, 2020
- A Proposal on Machine Learning via Dynamical SystemsCommunications in Mathematics and Statistics, 2017
- A Practical Guide to Applying Echo State Networks2012
- Information processing using a single dynamical node as complex systemNature Communications, 2011
- Reservoir computing approaches to recurrent neural network trainingComputer Science Review, 2009