Etienne Coutureau

Adaptive decision-making in dynamic environments requires flexible adjustment of learning speed to balance stability and flexibility. When outcomes are highly stochastic, learners must avoid over-interpreting noise and update more slowly, whereas in volatile environments where contingencies change frequently, learning should accelerate to rapidly incorporate new evidence. Theories propose that internal estimates of uncertainty tune learning rates through neuromodulation-dependent mechanisms. Here, we investigated how noradrenergic inputs from the locus coeruleus (LC) to the orbitofrontal cortex (OFC) support adaptive learning under uncertainty. We show that, in a probabilistic reversal learning task performed across different levels of stochasticity, rats exhibited behavior best explained by an adaptive reinforcement-learning model in which learning rates dynamically adjusted according to model-estimated stochasticity and volatility, outperforming standard fixed-rate models. Noradrenaline release in the OFC closely tracked trial-by-trial, model-derived volatility estimates around contingency changes. Disrupting LC→OFC noradrenergic inputs reproduced the model-predicted impairment in adaptive learning-rate adjustment associated with model-estimated volatility. Together, these findings identify OFC noradrenergic signaling as a key circuit mechanism supporting learning-rate adjustment in response to internal estimates of volatility during adaptive decision-making under uncertainty.

Adaptive decision-making in dynamic environments requires flexible adjustment of learning speed to balance stability and flexibility. When outcomes are highly stochastic, learners must avoid over interpreting noise and update more slowly, whereas in volatile environments where contingencies change frequently, learning should accelerate to rapidly incorporate new evidence. Theories propose that internal estimates of uncertainty tune learning rates through neuromodulatory-dependent mechanisms. Here, we investigated how noradrenergic inputs from the locus coeruleus (LC) to the orbitofrontal cortex (OFC) support adaptive learning under uncertainty. We show that rats performing a probabilistic reversal learning task exhibited behavior that was best explained by an adaptive reinforcement-learning model in which learning rates dynamically adjust according to estimated environmental volatility and stochasticity, outperforming standard fixed-rate models. Noradrenaline release in the OFC closely tracked trial-by-trial, model-derived volatility estimates around contingency changes. Disrupting LC→OFC noradrenergic inputs reproduced the model-predicted deficit in volatility-dependent adjustments of learning rate. Together, these findings identify OFC noradrenergic signaling as a key circuit mechanism for volatility-dependent modulation of learning rates during adaptive decision-making.

In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to stabilise learning and exploit what they currently believe to be their best option, or instead explore potential alternatives and learn fast from new observations. While such a trade‐off has been extensively studied in pretrained animals facing non‐stationary decision‐making tasks, it is yet unknown how they progressively tune it while learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long‐term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three‐armed bandit task across 24 days of pretraining. We found that the day‐by‐day evolution of rat performance and win‐shift tendency revealed a progressive stabilisation of the way they regulated reinforcement learning parameters. We successfully captured these behavioural adaptations using a meta‐learning model in which either the learning rate or the inverse temperature was controlled by the average reward rate.

A dynamic environment, such as the one we inhabit, requires organisms to continuously update their knowledge of the setting. While the prefrontal cortex is recognized for its pivotal role in regulating such adaptive behavior, the specific contribution of each prefrontal area remains elusive. In the current work, we investigated the direct involvement of two major prefrontal subregions, the medial prefrontal cortex (mPFC, A32D + A32V) and the orbitofrontal cortex (OFC, VO + LO), in updating pavlovian stimulus–outcome (S–O) associations following contingency degradation in male rats. Specifically, animals had to learn that a particular cue, previously fully predicting the delivery of a specific reward, was no longer a reliable predictor. First, we found that chemogenetic inhibition of mPFC, but not of OFC, neurons altered the rats’ ability to adaptively respond to degraded and non-degraded cues. Next, given the growing evidence pointing at noradrenaline (NA) as a main neuromodulator of adaptive behavior, we decided to investigate the possible involvement of NA projections to the two subregions in this higher-order cognitive process. Employing a pair of novel retrograde vectors, we traced NA projections from the locus ceruleus (LC) to both structures and observed an equivalent yet relatively segregated amount of inputs. Then, we showed that chemogenetic inhibition of NA projections to the mPFC, but not to the OFC, also impaired the rats’ ability to adaptively respond to the degradation procedure. Altogether, our findings provide important evidence of functional parcellation within the prefrontal cortex and point at mPFC NA as key for updating pavlovian S–O associations.

Obesity is associated with neurocognitive dysfunction, including memory deficits. This is particularly worrisome when obesity occurs during adolescence, a maturational period for brain structures critical for cognition. In rodent models, we recently reported that memory impairments induced by obesogenic high-fat diet (HFD) intake during the periadolescent period can be reversed by chemogenetic manipulation of the ventral hippocampus (vHPC). Here, we used an intersectional viral approach in HFD-fed male mice to chemogenetically inactivate specific vHPC efferent pathways to nucleus accumbens (NAc) or medial prefrontal cortex (mPFC) during memory tasks. We first demonstrated that HFD enhanced activation of both pathways after training and that our chemogenetic approach was effective in normalizing this activation. Inactivation of the vHPC–NAc pathway rescued HFD-induced deficits in recognition but not location memory. Conversely, inactivation of the vHPC–mPFC pathway restored location but not recognition memory impairments produced by HFD. Either pathway manipulation did not affect exploration or anxiety-like behaviour. These findings suggest that HFD intake throughout adolescence impairs different types of memory through overactivation of specific hippocampal efferent pathways and that targeting these overactive pathways has therapeutic potential.

A dynamic environment, such as the one we inhabit, requires organisms to continuously update their knowledge of the setting. While the prefrontal cortex is recognized for its pivotal role in regulating such adaptive behavior, the specific contributions of each prefrontal area remain elusive. In the current work, we investigated the direct involvement of two major prefrontal subregions, the medial prefrontal cortex (mPFC) and the ventrolateral orbitofrontal cortex (vlOFC), in updating Pavlovian stimulus-outcome (S-O) associations following contingency degradation. Specifically, animals had to learn that a specific cue, previously fully predicting the delivery of a specific reward, was no longer a reliable predictor. First, we found that chemogenetic inhibition of mPFC, but not of vlOFC, neurons altered the rats’ ability to adaptively respond to degraded and non-degraded cues. Next, given the growing evidence pointing at noradrenaline (NA) as a main neuromodulator of adaptive behavior, we decided to investigate the possible involvement of NA projections to the two subregions in this higher-order cognitive process. Employing a pair of novel retrograde vectors, we traced NA projections from the locus coeruleus (LC) to both structures and observed an equivalent yet relatively segregated amount of inputs. Then, we showed that chemogenetic inhibition of NA projections to the mPFC, but not to the vlOFC, also impaired the rats’ ability to adaptively respond to the degradation procedure. Altogether, our findings provide important evidence of functional parcellation within the prefrontal cortex and point at mPFC-NA as key for updating Pavlovian S-O associations. Significant statement: The ability to update stimulus-outcome (S-O) associations is a key adaptive behavior, essential for surviving in an ever-changing environment. The prefrontal cortex is well-known for playing a key role in this process. The discrete contribution of each prefrontal subregion and of different neurotransmitters, however, remains unclear. In the current study, we show that inhibiting medial prefrontal (mPFC), but not ventrolateral orbitofrontal cortex (vlOFC), neurons impairs the rats’ ability to update S-O associations following contingency degradation. Moreover, we demonstrate that discrete noradrenergic projections to the two subregions exist and that inhibiting the ones projecting to the mPFC, but not to the vlOFC, once again impairs the rats’ behavior, thereby implying a substantial contribution of noradrenaline in orchestrating this higher-order cognitive process.

In uncertain environments in which resources fluctuate continuously, animals must permanently decide whether to exploit what they currently believe to be their best option, or instead explore potential alternatives in case better opportunities are in fact available. While such a trade-off has been extensively studied in pretrained animals facing non-stationary decision-making tasks, it is yet unknown how they progressively tune it while progressively learning the task structure during pretraining. Here, we compared the ability of different computational models to account for long-term changes in the behaviour of 24 rats while they learned to choose a rewarded lever in a three-armed bandit task across 24 days of pretraining. We found that the day-by-day evolution of rat performance and win-shift tendency revealed a progressive stabilization of the way they regulated the exploration-exploitation trade-off. We successfully captured these behavioural adaptations using a meta-learning model in which the exploration-exploitation trade-off is controlled by the animal's average reward rate.

In a constantly changing environment, organisms must track the current relationship between actions and their specific consequences and use this information to guide decision-making. Such goal-directed behaviour relies on circuits involving cortical and subcortical structures. Notably, a functional heterogeneity exists within the medial prefrontal, insular, and orbitofrontal cortices (OFC) in rodents. The role of the latter in goal-directed behaviour has been debated, but recent data indicate that the ventral and lateral subregions of the OFC are needed to integrate changes in the relationships between actions and their outcomes. Neuromodulatory agents are also crucial components of prefrontal functions and behavioural flexibility might depend upon the noradrenergic modulation of the prefrontal cortex. Therefore, we assessed whether noradrenergic innervation of the OFC plays a role in updating action-outcome relationships in male rats. We used an identity-based reversal task and found that depletion or chemogenetic silencing of noradrenergic inputs within the OFC rendered rats unable to associate new outcomes with previously acquired actions. Silencing of noradrenergic inputs in the prelimbic cortex or depletion of dopaminergic inputs in the OFC did not reproduce this deficit. Together, our results suggest that noradrenergic projections to the OFC are required to update goal-directed actions.