Tom Verguts

Adaptive variability in behavior is crucial when environments promote unpredictability, yet its underlying cognitive mechanisms remain poorly understood. Volatile and adversarial environments can elicit variability by abrupt rule switches and explicitly reinforcing variable choices, respectively; however, literature lacks paradigms, computational models, and data to investigate and compare how (human) agents address such challenges. We introduce a novel hide-and-seek paradigm and use computational modeling to disentangle five candidate cognitive mechanisms of variability: stochastic choice, fast reinforcement learning, action-frequency-based memory, sequence-frequency-based memory, and episodic memory. Across three experiments (N = 290), we show that participants flexibly increased learning rate in volatile environments, but stochasticity, action-frequency-based memory, and reliance on episodic memory in adversarial environments. For the first time, these findings uncover that adaptive variability is not a unitary phenomenon but instead emerges from the flexible recruitment of multiple cognitive mechanisms as environmental demands intensify.

People adjust how fast they update task rules, depending on the volatility of their environment, by regulating cognitive control accordingly. We investigated whether this adaptation is primarily driven by recently experienced volatility in task demands, or can also be shaped by learned environmental knowledge indicating expected levels of volatility. To this end, we trained participants on a Wisconsin Card Sorting Test task where different environments required different speeds of task rule updating. We demonstrate that, initially, participants adopted task rule updating strategies depending on the most recently experienced levels of volatility and feedback (Experiment 1). However, after extensive (4 days) training (Experiment 2), participants updated task rules using strategies based on developed environmental knowledge. We also show how participants fine‐tuned their task rule updating strategies over learning. In sum, our findings provide important insights into how humans update task rules using different strategies and highlight the importance of multiday training in developing environmental knowledge for the regulation of cognitive control.

Different situations may require radically different information updating speeds (i.e., learning rates). Some demand fast learning rates while others benefit from using slower ones. To adjust learning rates, decision makers could rely on either global, meta-learned differences between environments, or faster but transient adaptations to locally experienced prediction errors. Here, we introduce a new paradigm that allows researchers to measure and empirically disentangle both forms of adaptation. Participants performed short blocks of trials of a continuous estimation task – fishing for crabs – on six different islands that required different optimal (initial) learning rates. Across two experiments, participants showed fast adaptations in learning rate within a block. Critically, participants also learned global environment-specific learning rates over the time course of the experiment, as evidenced by computational modelling and by the learning rates calculated on the very first trial when revisiting an environment (i.e., unconfounded by transient adaptations). Using representational similarity analyses of fMRI data, we found that differences in voxel pattern responses in the central orbitofrontal cortex (OFC) correlated with differences in these global environment-specific learning rates. Our findings show that humans adapt learning rates at both slow and fast time scales, and that the central OFC may support meta-learning by representing environment-specific task-relevant features such as learning rates.

Rhythmic fluctuations in neuronal excitability, called neural oscillations, pervade brain activity. If these oscillations have a functional role in behavior, they should also be expressed in a behavioral signature. However, identifying such behavioral signatures remains challenging and continues to motivate debate, particularly because aperiodic temporal structure can sometimes be mistaken for rhythmic activity. To assess behavioral rhythmicity while controlling for aperiodic dynamics, we applied a state-of-the-art autoregressive modelling approach to two datasets: one published dataset (final sample: n = 34), and a new dataset (final sample: n = 26) collected with an adapted dense-sampling design. In both datasets, AR modelling recovered reliable behavioral oscillations in the theta band. Reanalyses of the published dataset also reproduced the reported theta shift with task demands, with more difficult tasks showing slower theta. In contrast, despite clear rhythmicity, the new dataset showed no consistent frequency shift across conditions. Overall, this study establishes rigorous and robust detection of behavioral oscillations. It also suggests that the frequency modulation of behavioral oscillations by task difficulty requires further study.

Different situations may require radically different information updating speeds (i.e., learning rates). Some demand fast learning rates, while others benefit from using slower ones. To adjust learning rates, decision makers could rely on either global, meta-learned differences between environments, or faster but transient adaptations to locally experienced prediction errors. Here, we introduce a new paradigm that allows researchers to measure and empirically disentangle both forms of adaptations. Participants performed short blocks of trials of a continuous estimation task – fishing for crabs – on six different islands that required different optimal (initial) learning rates. Across two experiments, participants showed fast adaptations in learning rate within a block. Critically, participants also learned global environment-specific learning rates over the time course of the experiment, as evidenced by computational modelling and by the learning rates calculated on the very first trial when revisiting an environment (i.e., unconfounded by transient adaptations). Using representational similarity analyses of fMRI data, we found that differences in voxel pattern responses in the central orbitofrontal cortex correlated with differences in these global environment-specific learning rates. Our findings show that humans adapt learning rates at both slow and fast time scales, and that the central orbitofrontal cortex may support meta-learning by representing environment-specific task-relevant features such as learning rates.

Adaptively variable behavior can be advantageous in various fields such as sports (unpredictability), art (creativity), science (innovation), and problem-solving (thinking outside the box). Although previous studies identified experimental conditions under which humans and non-human animals show increased variable decision making, we have only a limited understanding of its underlying cognitive mechanisms. Using a reinforcement learning model, we simulate three different theorized strategies in an adversarial reward learning environment that requires very high variability. These mechanisms are (1) relying on a stochastic generator, (2) increasing one’s learning rate, or (3) upvaluing unchosen actions. A systematic parameter search and a policy gradient meta-learning algorithm both show that agents can adaptively increase variability with each of those strategies. Next, we fitted our model on existing datasets with humans, pigeons, and rats as subjects in adversarial environments. While all three species can engage in highly variable behavior, we find that only humans upvalue unchosen actions as a strategy to achieve variability.

Humans are remarkably efficient in exploring vast decision spaces. However, environments vary in how they are structured, raising the question of whether and how people adapt their generalization and exploration strategies accordingly. People might flexibly adapt their behavior within each environment (i.e., local adaptation), or also learn to associate specific strategies with distinct environments (i.e., meta-learning). Using a spatially correlated bandit paradigm and computational modeling, we examined how humans adapt their search strategy across two environments optimized for different levels of generalization. Across three experiments, participants showed more generalization and more random exploration in environments with stronger spatial correlations. Moreover, they meta-learned to associate different degrees of generalization with different environmental cues. These findings illustrate how humans control and adapt search strategies across diverse decision spaces.

Humans are remarkably efficient at adapting to different contextual demands by exerting optimal levels of cognitive flexibility versus stability for switching between different tasks. However, empirical findings document that these control adaptation processes can occur across multiple timescales, suggesting that different computational mechanisms may be involved. Here, we developed a recurrent neural network model to simulate behavioral indices of cognitive flexibility versus stability and investigated its dynamics. Throughout four simulation studies, we revealed that control adaptations that benefit from recent control states (activation-based) can support fast adjustments in cognitive flexibility, whereas control adaptations that depend on changes in control settings (weight-based) evolve more slowly over learning. Interestingly, like humans, the model further learned to associate contextual features to different control settings and use this knowledge to shift along a flexibility-stability continuum when encountering these same contexts. We further verified model-specific predictions through an analysis of an existing human dataset (N = 102). In sum, our work integrates fast and slow control adaptations within a unified framework and advances our understanding on human adaptative behavior and its regulatory mechanisms through the lens of activation-based versus weight-based adaptations.

Rhythmic fluctuations in neuronal excitability, called neural oscillations, pervade brain activity. If these oscillations have a functional role in behavior, they should also be expressed in a behavioral signature. However, identifying such a behavioral signature has remained controversial: evidence for behavioral oscillations is not yet firmly established, and aperiodic components can be misidentified as rhythmic. To test their existence while controlling for aperiodic dynamics, we applied a state-of-the-art autoregressive (AR) approach on two datasets: one published dataset, and a new dataset collected with a revised dense-sampling design. In both datasets, AR modeling recovered reliable behavioral oscillations in the theta band. Reanalyses of the published dataset also reproduced the reported theta shift with task demands, with more difficult tasks showing slower theta. In contrast, despite clear rhythmicity, the new dataset showed no consistent frequency shift across conditions. Overall, this study establishes rigorous and robust detection of behavioral oscillations. It also suggests that the frequency modulation of behavioral oscillations by task difficulty requires further study.

Flexible adaptation requires inferring the causes of feedback and adjusting accordingly. When tasks are hierarchically structured with high-level rules governing low-level perceptual judgements, good performance requires hierarchical reasoning. Yet the computational and neural mechanisms underlying this process remain unclear. Here, building on Bayesian modelling and recurrent neural network simulation of hierarchical reasoning, we verified that such algorithm and representations are implemented in human dorsal anterior cingulate cortex (dACC). dACC encodes accumulated errors and perceptual difficulty to estimate high-level rule switch confidence. These two variables were represented along two separable dimensions, and could be approximated by Gaussian basis functions. Orthogonality between these two dimensions yielded an optimal two-dimensional representation for hierarchical reasoning. A closer-to-orthogonal representational geometry also predicted better performance. Together, these findings provide an integrated computational and neural representational basis for hierarchical reasoning.