Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks
Open Access
- 27 August 2024
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 15 (1) , 1-15
Abstract
Physical neuromorphic computing, exploiting the complex dynamics of physical systems, has seen rapid advancements in sophistication and performance. Physical reservoir computing, a subset of neuromorphic computing, faces limitations due to its reliance on single systems. This constrains output dimensionality and dynamic range, limiting performance to a narrow range of tasks. Here, we engineer a suite of nanomagnetic array physical reservoirs and interconnect them in parallel and series to create a multilayer neural network architecture. The output of one reservoir is recorded, scaled and virtually fed as input to the next reservoir. This networked approach increases output dimensionality, internal dynamics and computational performance. We demonstrate that a physical neuromorphic system can achieve an overparameterised state, facilitating meta-learning on small training sets and yielding strong performance across a wide range of tasks. Our approach’s efficacy is further demonstrated through few-shot learning, where the system rapidly adapts to new tasks.Keywords
All Related Versions
Funding Information
- RCUK | Engineering and Physical Sciences Research Council (EP/W524335/1)
- Royal Academy of Engineering
This publication has 57 references indexed in Scilit:
- Nanomagnonics with artificial spin icePhysics Letters A, 2021
- DeePr-ESN: A deep projection-encoding echo-state networkInformation Sciences, 2020
- A Pure 2H‐MoS2 Nanosheet‐Based Memristor with Low Power Consumption and Linear Multilevel Storage for Artificial Synapse EmulatorAdvanced Electronic Materials, 2020
- Gradient descent optimizes over-parameterized deep ReLU networksMachine Learning, 2019
- Recent advances in physical reservoir computing: A reviewNeural Networks, 2019
- Meta-Learning2019
- Design of deep echo state networksNeural Networks, 2018
- Deep reservoir computing: A critical experimental analysisNeurocomputing, 2017
- Echo State Property of Deep Reservoir Computing NetworksCognitive Computation, 2017
- A Practical Guide to Applying Echo State Networks2012