Reinforcement-Based Decision Making in Corticostriatal Circuits: Mutual Constraints by Neurocomputational and Diffusion Models
Authors:
- 1 May 2012
- journal article
- Published by MIT Press in Neural Computation
- Vol. 24 (5) , 1186-1229
Abstract
In this letter, we examine the computational mechanisms of reinforce-ment-based decision making. We bridge the gap across multiple levels of analysis, from neural models of corticostriatal circuits—the basal ganglia (BG) model (Frank, 2005 , 2006 ) to simpler but mathematically tractable diffusion models of two-choice decision making. Specifically, we generated simulated data from the BG model and fit the diffusion model (Ratcliff, 1978 ) to it. The standard diffusion model fits underestimated response times under conditions of high response and reinforcement conflict. Follow-up fits showed good fits to the data both by increasing nondecision time and by raising decision thresholds as a function of conflict and by allowing this threshold to collapse with time. This profile captures the role and dynamics of the subthalamic nucleus in BG circuitry, and as such, parametric modulations of projection strengths from this nucleus were associated with parametric increases in decision boundary and its modulation by conflict. We then present data from a human reinforcement learning experiment involving decisions with low- and high-reinforcement conflict. Again, the standard model failed to fit the data, but we found that two variants similar to those that fit the BG model data fit the experimental data, thereby providing a convergence of theoretical accounts of complex interactive decision-making mechanisms consistent with available data. This work also demonstrates how to make modest modifications to diffusion models to summarize core computations of the BG model. The result is a better fit and understanding of reinforcement-based choice data than that which would have occurred with either model alone.Keywords
This publication has 68 references indexed in Scilit:
- Subthalamic nucleus stimulation reverses mediofrontal influence over decision thresholdNature Neuroscience, 2011
- Individual differences, aging, and IQ in two-choice tasksCognitive Psychology, 2010
- Frontal theta links prediction errors to behavioral adaptation in reinforcement learningNeuroImage, 2010
- Perceptual discrimination in static and dynamic noise: The temporal relation between perceptual encoding and decision making.Journal of Experimental Psychology: General, 2010
- Instructional control of reinforcement learning: A behavioral and neurocomputational investigationBrain Research, 2009
- Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitationNature Neuroscience, 2009
- A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewalPsychopharmacology, 2009
- Decision-making with multiple alternativesNature Neuroscience, 2008
- Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learningProceedings of the National Academy of Sciences, 2007
- Modeling response signal and response time data☆Cognitive Psychology, 2006