Origin of non-zero overlap in Hopfield neural networks beyond storage capacity
- 3 February 2026
- journal article
- Published by Springer Nature in Cognitive Neurodynamics
- Vol. 20 (1) , 39
Abstract
Hopfield networks are widely used models of associative memory. When the number of stored patterns exceeds the network's storage capacity, theoretical predictions show that the overlap between final states and memorized patterns should vanish. However, numerical simulations show that a small, non-zero overlap persists, indicating that the network retains residual memory. To investigate the origin of this phenomenon, we analyze the network's dynamics during the initial update steps. Using a signal-to-noise-ratio analysis, we demonstrate that when a node undergoes a state flip, the signal term of its neighbors is enhanced by the connecting link. This effect improves the stability of these neighboring neurons, facilitating a fraction of the network to remain aligned with the memory pattern and preventing a total loss of memory. Our findings elucidate the mechanism by which residual memory traces emerge in Hopfield networks beyond the storage limit.Keywords
Funding Information
- NSFC (11675096)
- The National Key R & D Program of China (2021ZD0201300)
This publication has 42 references indexed in Scilit:
- The computational foundations of dynamic coding in working memoryTrends in Cognitive Sciences, 2024
- Memristive neural network circuit implementation of associative learning with overshadowing and blockingCognitive Neurodynamics, 2022
- An engram of intentionally forgotten informationNature Communications, 2021
- Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imagingNature Communications, 2021
- The Biology of Forgetting—A PerspectiveNeuron, 2017
- On a Model of Associative Memory with Huge Storage CapacityJournal of Statistical Physics, 2017
- The form of the forgetting curve and the fate of memoriesJournal of Mathematical Psychology, 2011
- Self-consistent signal-to-noise analysis of Hopfield model with unit replacementNeural Networks, 2010
- Statistical mechanics of neural networks near saturationAnnals of Physics, 1987
- Learning and pattern recognition in spin glass modelsZeitschrift für Physik B Condensed Matter, 1985