Formalizing the Function of Anterior Insula in Rapid Adaptation
Open Access
- 4 December 2018
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Integrative Neuroscience
- Vol. 12, 61
Abstract
Anterior insula (aIns) is thought to play a crucial role in rapid adaptation in an ever-changing environment. Mathematically, it is known to track risk and surprise. Modern theories of learning, however, assign a dominant role to signed prediction errors (PEs), not to risk and surprise. Risk and surprise only enter to the extent that they modulate the learning rate, in an attempt to approximate Bayesian learning. Even without such modulation, adaptation is still possible, albeit slow. Here, I propose a new theory of learning, reference-model based learning (RMBL), where risk and surprise are central, and PEs play a secondary, though still crucial, role. The primary goal is to bring outcomes in line with expectations in the reference model (RM). Learning is modulated by how large the PEs are relative to model anticipation, i.e., to surprise as defined by the RM. In a target location prediction task where participants were continuously required to adapt, choices appeared to be closer with to RMBL predictions than to Bayesian learning. aIns reaction to surprise was more acute in the more difficult treatment, consistent with its hypothesized role in metacognition. I discuss links with related theories, such as Active Inference, Actor-Critic Models and Reference-Model Based Adaptive Control.Keywords
This publication has 39 references indexed in Scilit:
- The neural basis of metacognitive abilityPhilosophical Transactions of the Royal Society B: Biological Sciences, 2012
- An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing EnvironmentThe Journal of Neuroscience, 2010
- States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement LearningNeuron, 2010
- Neural correlates of risk prediction error during reinforcement learning in humansNeuroImage, 2009
- Reinforcement Learning or Active Inference?PLOS ONE, 2009
- How do you feel — now? The anterior insula and human awarenessNature Reviews Neuroscience, 2009
- Prediction of human errors by maladaptive changes in event-related brain networksProceedings of the National Academy of Sciences, 2008
- A Neural Substrate of Prediction and RewardScience, 1997
- On the Consistency of Bayes EstimatesThe Annals of Statistics, 1986
- Understanding the Kalman FilterThe American Statistician, 1983