Habits as action sequences: hierarchical action control and changes in outcome value
- 5 November 2014
- journal article
- Published by The Royal Society in Philosophical Transactions of the Royal Society B: Biological Sciences
- Vol. 369 (1655) , 20130482
Abstract
Goal-directed action involves making high-level choices that are implemented using previously acquired action sequences to attain desired goals. Such a hierarchical schema is necessary for goal-directed actions to be scalable to real-life situations, but results in decision-making that is less flexible than when action sequences are unfolded and the decision-maker deliberates step-by-step over the outcome of each individual action. In particular, from this perspective, the offline revaluation of any outcomes that fall within action sequence boundaries will be invisible to the high-level planner resulting in decisions that are insensitive to such changes. Here, within the context of a two-stage decision-making task, we demonstrate that this property can explain the emergence of habits. Next, we show how this hierarchical account explains the insensitivity of over-trained actions to changes in outcome value. Finally, we provide new data that show that, under extended extinction conditions, habitual behaviour can revert to goal-directed control, presumably as a consequence of decomposing action sequences into single actions. This hierarchical view suggests that the development of action sequences and the insensitivity of actions to changes in outcome value are essentially two sides of the same coin, explaining why these two aspects of automatic behaviour involve a shared neural structure.Keywords
This publication has 47 references indexed in Scilit:
- Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically OrganizedPLOS Computational Biology, 2013
- Speed/Accuracy Trade-Off between the Habitual and the Goal-Directed ProcessesPLOS Computational Biology, 2011
- Choice and contingency in the development of behavioral autonomy during instrumental conditioning.Journal of Experimental Psychology: Animal Behavior Processes, 2010
- Cortical and basal ganglia contributions to habit learning and automaticityTrends in Cognitive Sciences, 2010
- Hierarchically organized behavior and its neural foundations: A reinforcement learning perspectiveCognition, 2009
- Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual ActionNeuropsychopharmacology, 2009
- Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral controlNature Neuroscience, 2005
- Postconditioning devaluation of a reinforcer affects instrumental responding.Journal of Experimental Psychology: Animal Behavior Processes, 1985
- Variations in the Sensitivity of Instrumental Responding to Reinforcer DevaluationQuarterly Journal of Experimental Psychology, 1982
- Cognitive maps in rats and men.Psychological Review, 1948