Reinforcement learning when your life depends on it: A neuro-economic theory of learning
Open Access
- 28 October 2024
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 20 (10) , e1012554
Abstract
Synaptic plasticity enables animals to adapt to their environment, but memory formation can require a substantial amount of metabolic energy, potentially impairing survival. Hence, a neuro-economic dilemma arises whether learning is a profitable investment or not, and the brain must therefore judiciously regulate learning. Indeed, in experiments it was observed that during starvation, Drosophila suppress formation of energy-intensive aversive memories. Here we include energy considerations in a reinforcement learning framework. Simulated flies learned to avoid noxious stimuli through synaptic plasticity in either the energy expensive long-term memory (LTM) pathway, or the decaying anesthesia-resistant memory (ARM) pathway. The objective of the flies is to maximize their lifespan, which is calculated with a hazard function. We find that strategies that switch between the LTM and ARM pathways, based on energy reserve and reward prediction error, prolong lifespan. Our study highlights the significance of energy-regulation of memory pathways and dopaminergic control for adaptive learning and survival. It might also benefit engineering applications of reinforcement learning under resources constraints.Keywords
All Related Versions
Funding Information
- Nottingham Vice Chancellor’s International Scholarship (R00700)
This publication has 30 references indexed in Scilit:
- Spaced Training Forms Complementary Long-Term Memories of Opposite Valence in DrosophilaNeuron, 2020
- Coordinated and Compartmentalized Neuromodulation Shapes Sensory Processing in DrosophilaCell, 2015
- Heterosynaptic Plasticity Underlies Aversive Olfactory Learning in DrosophilaNeuron, 2015
- Activity of Defined Mushroom Body Output Neurons Underlies Learned Olfactory Behavior in DrosophilaNeuron, 2015
- Reinforcement signalling in Drosophila; dopamine does it all after allCurrent Opinion in Neurobiology, 2013
- Parallel Processing of Appetitive Short- and Long-Term Memories In DrosophilaCurrent Biology, 2011
- A neoHebbian framework for episodic memory; role of dopamine-dependent late LTPTrends in Neurosciences, 2011
- Traces of Drosophila MemoryNeuron, 2011
- Glucose enhancement of human memory: A comprehensive research review of the glucose memory facilitation effectNeuroscience & Biobehavioral Reviews, 2011
- Genetic dissection of consolidated memory in DrosophilaCell, 1994