Time representation in reinforcement learning models of the basal ganglia
Open Access
- 1 January 2014
- journal article
- review article
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 7, 194
Abstract
Reinforcement learning models have been influential in understanding many aspects of basal ganglia function, from reward prediction to action selection. Time plays an important role in these models, but there is still no theoretical consensus about what kind of time representation is used by the basal ganglia. We review several theoretical accounts and their supporting evidence. We then discuss the relationship between reinforcement learning models and the timing mechanisms that have been attributed to the basal ganglia. We hypothesize that a single computational system may underlie both reinforcement learning and interval timing—the perception of duration in the range of seconds to hours. This hypothesis, which extends earlier models by incorporating a time-sensitive action selection mechanism, may have important implications for understanding disorders like Parkinson's disease in which both decision making and timing are impaired.Keywords
This publication has 69 references indexed in Scilit:
- Dissociating hippocampal and striatal contributions to sequential prediction learningEuropean Journal of Neuroscience, 2012
- A Model of Interval Timing by Neural IntegrationThe Journal of Neuroscience, 2011
- Population clocks: motor timing with neural dynamicsTrends in Cognitive Sciences, 2010
- Neuroanatomical and Neurochemical Substrates of TimingNeuropsychopharmacology, 2010
- Neural representation of time in cortico-basal ganglia circuitsProceedings of the National Academy of Sciences, 2009
- Neurocomputational models of basal ganglia function in learning, memory and choiceBehavioural Brain Research, 2009
- A Role for Dopamine in Temporal Decision Making and Reward Maximization in ParkinsonismJournal of Neuroscience, 2008
- Ketamine “unlocks” the reduced clock-speed effects of cocaine following extended training: Evidence for dopamine–glutamate interactions in timing and time perceptionNeurobiology of Learning and Memory, 2007
- What makes us tick? Functional and neural mechanisms of interval timingNature Reviews Neuroscience, 2005
- A Neural Substrate of Prediction and RewardScience, 1997