George Elliott Wimmer

A core premise in decision-making research is that beliefs about uncertain outcomes shape value-based choice: pessimistic expectations should reduce goal pursuit, whereas optimistic expectations should promote it. Depression provides a critical test of this assumption, as motivational symptoms such as apathy and anhedonia are linked to diminished expectations of success. Yet whether such biases actually impair goal-directed behavior, particularly when risk must be learned from experience, remains unknown. We developed a multi-step planning task in which participants (n = 384) learned the structure and risks of maze-like environments across two days, then chose between certain rewards and risky goals varying in reward magnitude and distance, with greater distance entailing higher compounding risk. Using transdiagnostic symptom factors, we observed a dissociation: individuals with elevated apathy-anhedonia reported lower subjective expectations of goal success, yet showed no reduction in risky goal pursuit. Instead, higher apathy-anhedonia was associated with enhanced performance, including stronger discrimination of goal values, increased model-based flexibility, and faster goal navigation. These effects were specific to apathy-anhedonia; other symptom factors, including worry and impulsivity, were unrelated to expectations or goal-directed behavior. Together, these findings demonstrate that pessimistic beliefs and goal-directed competence are dissociable, revealing that motivational symptoms can coexist with sharpened value-based planning when risk is learned from experience.

Impairments in reward-related behavior in everyday life are characteristic of major depressive disorder and closely linked with anhedonia. While previous research has suggested a negative relationship between reward-related behavior and symptoms of depression or anhedonia, findings across clinical and non-clinical studies are mixed. We conducted three online studies, recruiting a combined 629 nonpatient online participants. The studies consisted of an exploratory study, a preregistered replication, and data from a separate study utilizing a distinct learning task. Participants completed reinforcement learning tasks where they learned to select either the more rewarding response or more rewarding stimulus, as well as a large set of self-report measures of symptoms and traits. Contrary to expectations, we observed a positive association between reinforcement learning performance and anhedonia symptoms across all three studies. General depression symptoms showed one positive association and two null associations with performance. Our results contrast with the assumption of a negative association between reinforcement learning behavior and symptoms of depression or anhedonia. Instead, our results suggest that in large non-selective samples, anhedonia symptoms can be positively associated with reward-guided learning and decision-making.

Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making (“planning”) and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.

Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decisionmaking (‘planning’) and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography (MEG) and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods. Our results provide new support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by on-going computational and task demands. Significance statement: The sequential neural reactivation of prior experience, known as replay, is considered to be an important mechanism for both future planning and preserving memories of the past. Whether, and how, replay supports both of these functions remains unknown. Here, in humans, we found that prior to a choice, rapid replay of potential future paths was enhanced when planning was more beneficial. By contrast, after choice feedback, when no future actions are imminent, we found evidence for a memory preservation signal evident in enhanced replay of paths that had been visited less in the recent past. The results demonstrate that distinct replay signatures, expressed at different times, relate to two dissociable cognitive functions.

Neuroscience research has illuminated the mechanisms supporting learning from reward feedback, demonstrating a critical role for the striatum and midbrain dopamine system. However, in humans, short-term working memory that is dependent on frontal and parietal cortices can also play an important role, particularly in commonly used paradigms in which learning is relatively condensed in time. Given the growing use of reward-based learning tasks in translational studies in computational psychiatry, it is important to understand the extent of the influence of working memory and also how core gradual learning mechanisms can be better isolated. In our experiments, we manipulated the spacing between repetitions along with a post-learning delay preceding a test phase. We found that learning was slower for stimuli repeated after a long delay (spaced-trained) compared to those repeated immediately (massed-trained), likely reflecting the remaining contribution of feedback learning mechanisms when working memory is not available. For massed learning, brief interruptions led to drops in subsequent performance, and individual differences in working memory capacity positively correlated with overall performance. Interestingly, when tested after a delay period but not immediately, relative preferences decayed in the massed condition and increased in the spaced condition. Our results provide additional support for a large role of working memory in reward-based learning in temporally condensed designs. We suggest that spacing training within or between sessions is a promising approach to better isolate and understand mechanisms supporting gradual reward-based learning, with particular importance for understanding potential learning dysfunctions in addiction and psychiatric disorders.

Aversive and rewarding experiences can exert a strong influence on subsequent behavior. While decisions are often supported by the value of single past episodes, most research has focused on the role of well-learned value associations. Recent studies have begun to investigate the influence of reward-associated episodes, but it is unclear if these results generalize to negative experiences such as pain. To investigate whether and how the value of previous aversive experiences modulates behavior and brain activity, in our experiments female and male human participants experienced episodes of high or low pain in conjunction with incidental, trial-unique neutral pictures. In an incentive-compatible surprise test phase, we found that participants avoided pain-paired objects. In a separate fMRI experiment, at test, participants exhibited significant pain value memory. Neurally, when participants were re-exposed to pain-paired objects, we found no evidence for reactivation of pain-related patterns in pain-responsive regions such as the anterior insula. Critically, however, we found significant reactivation of pain-related patterns of activity in the hippocampus, such that activity significantly discriminated high versus low pain episodes. Further, stronger reactivation in the anterior hippocampus was related to improved pain value memory performance. Our results demonstrate that single incidental aversive experiences can build memories that affect decision making and that this influence may be supported by the hippocampus. SIGNIFICANCE STATEMENT: Aversive and rewarding experiences can exert a strong influence on our subsequent behavior. While decisions are often supported by single past negative or positive episodes, most research has focused on the role of well-learned value associations. In experiments using aversive heat pain in conjunction with incidental objects, we found that participant’s choices were biased by the level of pain associated with the objects. Further, when participants saw the objects again, pain-related neural patterns in the hippocampus were re-expressed and this was related to pain value memory performance. These results suggest a mechanism by which even single negative experiences can guide our later decisions.

Retrieval of everyday experiences is fundamental for informing our future decisions. The fine-grained neurophysiological mechanisms that support such memory retrieval are largely unknown. We studied participants who first experienced, without repetition, unique multicomponent 40–80-s episodes. One day later, they engaged in cued retrieval of these episodes while undergoing magnetoencephalography. By decoding individual episode elements, we found that trial-by-trial successful retrieval was supported by the sequential replay of episode elements, with a temporal compression factor of >60. The direction of replay supporting retrieval, either backward or forward, depended on whether the task goal was to retrieve elements of an episode that followed or preceded, respectively, a retrieval cue. This sequential replay was weaker in very-high-performing participants, in whom instead we found evidence for simultaneous clustered reactivation. Our results demonstrate that memory-mediated decisions are supported by a rapid replay mechanism that can flexibly shift in direction in response to task goals.

Aversive and rewarding experiences can exert a strong influence on subsequent behavior. While decisions are often supported by the value of single past episodes, most research has focused on the role of well-learned value associations. Recent studies have begun to investigate the influence of reward-associated episodes, but it is unclear if these results generalize to negative experiences such as pain. To investigate whether and how the value of previous aversive experiences modulates behavior and brain activity, in our experiments female and male human participants experienced episodes of high or low pain in conjunction with incidental, trial-unique neutral pictures. In an incentive-compatible surprise test phase, we found that participants avoided pain-paired objects. In a separate fMRI experiment, at test, participants exhibited significant pain value memory. Neurally, when participants were re-exposed to pain-paired objects, we found no evidence for reactivation of pain-related patterns in pain-responsive regions such as the anterior insula. Critically, however, we found significant reactivation of pain-related patterns of activity in the hippocampus, such that activity significantly discriminated high versus low pain episodes. Further, stronger reactivation in the anterior hippocampus was related to improved pain value memory performance. Our results demonstrate that single incidental aversive experiences can build memories that affect decision making and that this influence may be supported by the hippocampus. Significance Statement: Aversive and rewarding experiences can exert a strong influence on our subsequent behavior. While decisions are often supported by single past negative or positive episodes, most research has focused on the role of well-learned value associations. In experiments using aversive heat pain in conjunction with incidental objects, we found that participant’s choices were biased by the level of pain associated with the objects. Further, when participants saw the objects again, pain-related neural patterns in the hippocampus were re-expressed and this was related to pain value memory performance. These results suggest a mechanism by which even single negative experiences can guide our later decisions.

Neuroscience research has illuminated the mechanisms supporting learning from reward feedback, demonstrating a critical role for the striatum and midbrain dopamine system. However, in humans, short-term working memory that is dependent on frontal and parietal cortices can also play an important role, particularly in commonly-used paradigms in which learning is relatively condensed in time. Given the growing use of reward-based learning tasks in translational studies in computational psychiatry, it is important to understand the degree of working memory contributions and whether gradual learning mechanisms can be better isolated. In our experiments, we manipulated the spacing between repetitions along with a post-learning delay preceding a test phase. We found that learning was slower for stimuli repeated after a long delay (spaced-trained) compared to those repeated immediately (massed-trained), likely reflecting the remaining contribution of feedback learning mechanisms when working memory is not available. Brief interruptions of massed learning led to drops in subsequent performance. Further, individual differences in working memory capacity positively correlated with massed learning performance. Critically, after a delay period but not immediately, relative preferences decayed in the massed condition and increased in the spaced condition. Overall, our results provide additional support for a large role of working memory in reward-based learning in temporally condensed designs. We suggest that spacing training within or between sessions is a promising approach to better isolate and understand mechanisms supporting gradual reward-based learning, with particular importance for understanding potential learning dysfunctions in addiction and psychiatric disorders.

Memory for everyday experience shapes our representation of the structure of the world, while retrieval of these experiences is fundamental for informing our future decisions. The fine-grained neurophysiological mechanisms that support such retrieval are largely unknown. We studied participants who first experienced, without repetition, unique multi-component episodes. One day later, they engaged in cued retrieval of these episodes whilst undergoing magnetoencephalography (MEG). By decoding individual episode elements, we found that trial-by-trial successful retrieval was supported by sequential replay of episode elements, with a temporal compression factor greater than 60. The direction of replay supporting this retrieval, either backward or forward, depended on whether a participant’s goal was to retrieve elements of an episode that followed or preceded a retrieval cue, respectively. This sequential replay was weaker in very high performing participants, where instead we found evidence for simultaneous clustered reactivation. Our results demonstrate that memory-mediated decisions are supported by a rapid replay mechanism that can flexibly shift in direction in response to task requirements. One Sentence Summary: Recall of extended episodes of experience is supported by compressed replay of memory elements that flexibly changes direction depending on task temporal orientation.