Maël Lebreton

In many circumstances, choices result in multiple simultaneous outcomes, all of which should be integrated to optimally update reward expectation. Yet, to date, empirical investigations of reinforcement learning have mostly focused on situations where choices deliver only one outcome at a time. To understand how humans learn from multiple outcomes, we designed a new reinforcement learning task, where the selection of an option resulted in two outcomes drawn from the same underlying distribution over potential gains and losses, even if ultimately only one of the two counted for the final payoff. Behavioral results show that the two - equally informative- outcomes are considered, yet asymmetrically as function of their valence. This behavioral observation is backed up by computational analyses showing that, on top of previously documented asymmetric update, multiple outcome integration is biased, such as it overweighs rewards over punishments. Behavioral and computational results were paralleled by eye-tracking analysis showing that attention deployment is biased by outcome valence and relevance. The main results were confirmed in a second experiment featuring complete feedback information. Overall, our findings suggest that, when options deliver multiple discordant outcomes, losses tend to be neglected compared to gains.

Risk preferences are shaped by outcome anticipation and feedback integration. Understanding how these processes evolve across the human lifespan is essential for theoretical models of decision-making, for applications in education, and behavioral public policy aimed at protecting against age-related vulnerabilities. This study examines the lifespan development (N = 658; ages 8 to 78) of decision-making under risk through the lens of these two constitutive components, using a behavioral paradigm merging descriptive and experiential information. Our findings replicate the established pattern of a peak of risk-taking during adolescence, followed by a steady decline with aging. Moreover, our results show that while the influence of outcome anticipation matures gradually during childhood and adolescence, the propensity to learn from feedback diminishes progressively with age. These results suggest that while adults may rely on more sophisticated internal models of probability, these models may paradoxically support faulty reasoning, potentially contributing to suboptimal choices.

Whether individuals feel confident about their own actions, choices, or statements being correct, and how these confidence levels differ between individuals are two key primitives for countless behavioral theories and phenomena. In cognitive tasks, individual confidence is typically measured as the average of reports about choice accuracy, but how reliable is the resulting characterization of within- and between-individual confidence remains surprisingly undocumented. Here, we perform a large-scale resampling exercise in the Confidence Database (103 studies, 6000 participants) to investigate the reliability of individual confidence estimates, and of comparisons across individuals' confidence levels. Our results show that confidence estimates are more stable than their choice-accuracy counterpart, reaching a reliability plateau after roughly 50 trials, regardless of a number of task design characteristics. While constituting a reliability upper-bound for task-based confidence measures, and thereby leaving open the question of the reliability of the construct itself, these results characterize the robustness of past and future task designs.

The ubiquity of over-optimism, overconfidence, wishful thinking and desirability biases suggest that a universal motivational mechanism distorts all decisions and judgments uniformly. Here, we investigate this intuition, by assessing the relative effects of incentives on two types of probabilistic judgments: beliefs about externally generated hypotheses, versus confidence in one’s own decisions. Across four perceptual decision-making experiments, we manipulate the agency of decisions over ambiguous states-of-the-world and the monetary incentives for accurate probabilistic judgments about decision accuracy. Results show that incentives consistently bias participants’ reports, but, contrary to the uniform motivational bias intuition, substantially more for confidence than belief judgments. Modelling probabilistic judgments as an incentive-dependent weighting of evidence that is congruent with a (covert) decision rationalizes this apparent discrepancy. We conclude that gain versus loss prospects modulate how we integrate evidence confirming our endorsed hypothesis, shedding light on the formation of beliefs and confidence, and their interaction with motivational processes.

The ubiquity of over-optimism, overconfidence, wishful thinking and desirability biases suggest that a universal motivational mechanism distorts all decisions and judgments uniformly. Here, we investigate this intuition, by assessing the relative effects of incentives on two types of probabilistic judgments: beliefs about externally generated hypotheses, versus confidence in one’s own decisions. Across four perceptual decision-making experiments, we manipulate the agency of decisions over ambiguous states-of-the-world and the monetary incentives for accurate probabilistic judgments about decision accuracy. Results show that incentives consistently bias participants’ reports, but, contrary to the uniform motivational bias intuition, substantially more for confidence than belief judgments. Modelling probabilistic judgments as an incentive-dependent weighting of evidence that is congruent with a (covert) decision rationalizes this apparent discrepancy. We conclude that gain versus loss prospects modulate how we integrate evidence confirming our endorsed hypothesis, shedding light on the formation of beliefs and confidence, and their interaction with motivational processes.

Background and aims: Gambling Disorder (GD) is associated with maladaptive decision-making, possibly driven by biases in learning and confidence judgments. While prior research report abnormal learning rates and heightened overconfidence in GD, the affected cognitive mechanism producing these joint deficits has so far remained unidentified. Our study aims to fill this gap using a recently established reinforcement learning (RL) experimental and computational framework linking learning processes, outcome-valence effects and confidence judgments. Methods: We pre-registered and tested the hypotheses that GD patients exhibit increased (over)confidence and confirmatory learning bias, and increased outcome valence effects on choice accuracy and confidence judgements in in 18 participants with GD and 19 matched controls. Results: While our findings replicated the main behavioral patterns of choices and confidence judgments, and confirmed their computational foundations, we did not find any group differences between the controls and patients with GD. Discussion and Conclusions: The current findings speak to the inconsistent findings of abnormalities in confidence and learning in GD. Systematic research is necessary to better understand the influence of possibly mediating factors such as disorder-related idiosyncrasies (e.g. skill- vs chance-based preferences) to further clarify if, when and how confidence and learning are affected in people with GD.

Many real-life decisions involve a tension between short-term and long-term outcomes, which requires forward-looking abilities. In reinforcement learning, this tension arises at the initial stage of multi-step learning and decision tasks, where forward-looking decisions collect smaller immediate rewards but govern the transition to more advantageous second-stage states. Here, we investigated the neural mechanisms underlying such forward-looking decisions in a cohort of healthy participants undergoing fMRI scanning (N=28). Behavioral results confirmed that participants were able to learn concurrently reward values and state-transition probabilities. By contrasting BOLD signal elicited by first-stage versus second-stage at the time of decision, we isolated a brain network, with central nodes in the bilateral parahippocampal cortex (BA37), whose activity correlate with forward-looking choices. BOLD activity in another network, including the bilateral parietal cortex, correlated with structure learning signal at the time of outcome processing. Our results shed new light on the neural bases of model-based reinforcement learning by suggesting a specific role of the parahippocampal cortex in forward planning and the parietal cortex in learning the Markovian structure of the task.

According to the normative standpoint, decision-making under risk, as it is traditionally instantiated in choice among fully described probabilistic lotteries, should not be affected by the disclosure of outcomes (feedback). Yet, contrary to this normative prescription, empirical studies have repeatedly reported that risk preferences are affected by immediate presentation of choice outcomes. The consensual and intuitive hypothesis is that feedback affects risk preferences via a learning process, whereby experienced outcomes alter the representation of subjective values. However, despite a relatively large body of published studies, available evidence does not allow robustly establishing the direction and the impact of feedback on key decision variables, namely risk propensity and expected value maximization, limiting our ability to establish a clear cognitive mechanism. Here, we ran seven behavioral experiments, tailored to address this gap and found that the presence of feedback consistently increases risk-taking, without any detectable impact on expected value maximization. Crucially, fine-grained analyses of the temporal dynamics of the effect of feedback directly falsified one of the currently most influential models of the role of experience in decisions under risk, and challenges any instantiation of the learning hypotheses. These results rather favor of an attitudinal effect, induced by the anticipation of feedback information. Epistemic curiosity and regret avoidance may drive this effect in partial and complete feedback conditions, respectively.