Symmetry-aware reservoir computing
- 13 October 2021
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review E
- Vol. 104 (4) , 045307
Abstract
We demonstrate that matching the symmetry properties of a reservoir computer (RC) to the data being processed dramatically increases its processing power. We apply our method to the parity task, a challenging benchmark problem that highlights inversion and permutation symmetries, and to a chaotic system inference task that presents an inversion symmetry rule. For the parity task, our symmetry-aware RC obtains zero error using an exponentially reduced neural network and training data, greatly speeding up the time to result and outperforming artificial neural networks. When both symmetries are respected, we find that the network size necessary to obtain zero error for 50 different RC instances scales linearly with the parity-order . Moreover, some symmetry-aware RC instances perform a zero error classification with only for . Furthermore, we show that a symmetry-aware RC only needs a training data set with size on the order of to obtain such a performance, an exponential reduction in comparison to a regular RC which requires a training data set with size on the order of to contain all possible -bit-long sequences. For the inference task, we show that a symmetry-aware RC presents a normalized root-mean-square error three orders-of-magnitude smaller than regular RCs. For both tasks, our RC approach respects the symmetries by adjusting only the input and the output layers, and not by problem-based modifications to the neural network. We anticipate that the generalizations of our procedure can be applied in information processing for problems with known symmetries.
Keywords
All Related Versions
Funding Information
- Raytheon BBN Technologies (60150)
This publication has 26 references indexed in Scilit:
- Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from dataChaos: An Interdisciplinary Journal of Nonlinear Science, 2017
- Computing with networks of nonlinear mechanical oscillatorsPLOS ONE, 2017
- Reservoir observers: Model-free inference of unmeasured variables in chaotic systemsChaos: An Interdisciplinary Journal of Nonlinear Science, 2017
- N-bit Parity Neural Networks with minimum number of threshold neuronsOpen Engineering, 2016
- Computational capabilities of random automata networks for reservoir computingPhysical Review E, 2013
- Selection of Proper Neural Network Sizes and Architectures—A Comparative StudyIEEE Transactions on Industrial Informatics, 2012
- Real-Time Computation at the Edge of Chaos in Recurrent Neural NetworksNeural Computation, 2004
- Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless CommunicationScience, 2004
- Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on PerturbationsNeural Computation, 2002
- Birthday paradox, coupon collectors, caching algorithms and self-organizing searchDiscrete Applied Mathematics, 1992