Stability through plasticity: Finding robust memories through representational drift
Preprint
Abstract
Memories are believed to be stored in synapses and retrieved through the reactivation of neural ensembles. Learning alters synaptic weights, which can interfere with previously stored memories that share the same synapses, creating a tradeoff between plasticity and stability. Interestingly, neural representations exhibit significant dynamics, even in stable environments, without apparent learning or forgetting—a phenomenon known as representational drift. Theoretical studies have suggested that multiple neural representations can correspond to a memory, with post-learning exploration of these representation solutions driving drift. However, it remains unclear whether representations explored through drift differ from those learned or offer unique advantages. Here we show that representational drift uncovers noise-robust representations that are otherwise difficult to learn. We first define the non-linear solution space manifold of synaptic weights for a fixed input-output mapping, which allows us to disentangle drift from learning and forgetting and simulate representational drift as diffusion within this manifold. Solutions explored by drift have many inactive and saturated neurons, making them robust to weight perturbations due to noise or continual learning. Such solutions are prevalent and entropically favored by drift, but their lack of gradients makes them difficult to learn and non-conducive to further learning. To overcome this, we introduce an allocation procedure that selectively shifts representations for new information into a learning-conducive regime. By combining allocation with drift, we resolve the tradeoff between learnability and robustness.Keywords
This publication has 88 references indexed in Scilit:
- Organization of hippocampal CA3 into correlated cell assemblies supports a stable spatial codeCell Reports, 2023
- Inhibitory conductance controls place field dynamics in the hippocampusCell Reports, 2022
- Representational drift: Emerging theories for continual learning and experimental future directionsCurrent Opinion in Neurobiology, 2022
- Drifting neuronal representations: Bug or feature?Biological Cybernetics, 2022
- Theoretical principles for illuminating sensorimotor processing with brain-wide neuronal recordingsCurrent Opinion in Neurobiology, 2020
- Causes and consequences of representational driftCurrent Opinion in Neurobiology, 2019
- Dynamic Reorganization of Neuronal Activity Patterns in Parietal CortexCell, 2017
- Homeostatic synaptic plasticity: from single synapses to neural circuitsCurrent Opinion in Neurobiology, 2012
- The Self-Tuning Neuron: Synaptic Scaling of Excitatory SynapsesCell, 2008
- Neural Control of Input into Long Term Memory: Limbic System and Amnestic Syndrome in Man1973