Ari E. Kahn

Children are adept statistical learners, capable of parsing streams of structured input into meaningful units, but the cognitive processes they engage during learning may differ from those of adults. To date, however, it is unclear how learners of different ages predict upcoming experience when navigating environments with complex structure, as well as how changes in predictive learning mechanisms influence structured knowledge acquisition. To address this question, we tested 106 children, adolescents, and adults, ages 8–22 years, on a predictive learning task, in which they experienced sequences of stimuli with a higher‐order temporal structure. After an initial learning phase, participants’ explicit knowledge of the relations between stimuli was probed via two additional task measures. We used a recently introduced computational model to characterize participants’ response times during learning, and found that all participants relied on simple, recency‐based prediction, anticipating that they would encounter stimuli they recently encountered in the past. With increasing age, however, participants demonstrated greater evidence of additionally relying on a more sophisticated learning mechanism, which captured a predictive representation of the conditional relations between stimuli. Though predictive learning changed with age, we found only weak evidence that these changes related to the acquisition of explicit knowledge of the environment. Our results suggest that the learning mechanisms through which people parse continuous streams of experience change with age, influencing their predictions about upcoming events. Summary: Children, adolescents, and adults all learn to segment continuous streams of structured perceptual input, but they may do so via different learning processes. We examined how the learning mechanisms that enable people to predict upcoming experiences change and relate to structured knowledge acquisition across development. Computational modeling revealed that in a graph‐learning task, younger participants relied on simple, recency‐based prediction, while older participants tracked temporal relations between stimuli. The extent to which participants engaged in this sophisticated form of predictive learning only weakly related to their knowledge of the task's structure.

Children are adept statistical learners, capable of parsing streams of structured input into meaningful units, but the cognitive processes they engage during learning may differ from those of adults. To date, however, it is unclear how learners of different ages predict upcoming experience when navigating environments with complex structure, as well as how changes in predictive learning mechanisms influence structured knowledge acquisition. To address this question, we tested 106 children, adolescents, and adults, ages 8 - 22 years, on a predictive learning task, in which they experienced sequences of stimuli with a higher-order temporal structure. After an initial learning phase, participants’ explicit knowledge of the relations between stimuli was probed via two additional task measures. We used a recently introduced computational model to characterize participants’ response times during learning, and found that all participants relied on simple, recency-based prediction, anticipating that they would encounter stimuli they recently encountered in the past. With increasing age, however, participants demonstrated greater evidence of additionally relying on a more sophisticated learning mechanism, which captured a predictive representation of the conditional relations between stimuli. Though predictive learning changed with age, we found only weak evidence that these changes related to the acquisition of explicit knowledge of the environment. Our results suggest that the learning mechanisms through which people parse continuous streams of experience change with age, influencing their predictions about upcoming events.

Learning where and when rewards like food and water are available is essential for survival ^1,2 . In the simplest cases where resource availability is stable, animals can learn reward contingencies by integrating outcomes across repeated samples of each possible action. In more natural settings, however, reward availability is governed by structured higher-order rules such as depletion and repletion over time. To adapt flexibly to such changing environments, optimal choices require meta-learning wherein animals learn how to learn from external feedback, ultimately enabling them to infer the underlying reward structure from abstract, generalizable rules rather than relying solely on recent outcomes ^3,4 . The existence of meta-learning in animal behavior is well established ^3–8 , yet the neural circuits and computations that implement it remain poorly understood ^9–11 . Here we investigated meta-learning using a spatial foraging task in which rats acquired a depletion-repletion rule that regulated reward availability, and carried out longitudinal, high-density recordings from the medial prefrontal cortex (mPFC). We show that meta-learning engages specific, systematic changes in mPFC neural dynamics that embed the learned rule and thereby alter how the network learns action values from reward outcomes. These dynamics are based on mixed coding of task structure and value in individual mPFC neurons. At the population level, this coding organizes into low-dimensional dynamical motifs that generalize across task conditions. As meta-learning progresses, these motifs are reshaped to instantiate both rule-guided inference of future states before outcome delivery and rule-based value updating during the outcome period. These results indicate that meta-learning sculpts pre-existing prefrontal dynamics to support the acquisition of new, generalizable reward-learning strategies.

Decisions in humans and other organisms depend, in part, on learning and using models that capture the statistical structure of the world, including the long-run expected outcomes of our actions. One prominent approach to forecasting such long-run outcomes is the successor representation (SR), which predicts future states aggregated over multiple timesteps. Although much behavioral and neural evidence suggests that people and animals use such a representation, it remains unknown how they acquire it. It has frequently been assumed to be learned by temporal difference bootstrapping (SR-TD(0)), but this assumption has largely not been empirically tested or compared to alternatives including eligibility traces (SR-TD( λ > 0 )). Here we address this gap by leveraging trial-by-trial reaction times in graph sequence learning tasks, which are favorable for studying learning dynamics because the long horizons in these studies differentiate the transient update dynamics of different learning rules. We examined the behavior of SR-TD( λ ) on a probabilistic graph learning task alongside a number of alternatives, and found that behavior was best explained by a hybrid model which learned via SR-TD( λ ) alongside an additional predictive model of recency. The relatively large λ we estimate indicates a predominant role of eligibility trace mechanisms over the bootstrap-based chaining typically assumed. Our results provide insight into how humans learn predictive representations, and demonstrate that people simultaneously learn the SR alongside lower-order predictions.

By building a mental model of how the world works and using it to forecast the outcomes of different actions, a learner can make flexible choices in changing environments. However, while children and adolescents readily acquire structured knowledge about their environments, relative to adults, they tend to demonstrate weaker signatures of leveraging this knowledge to plan actions. One explanation for these developmental differences is that using a mental model to prospectively simulate potential choices and their outcomes is computationally costly, taxing cognitive control and working memory mechanisms that continue to develop into adulthood. Here, we ask whether children might effectively leverage structured knowledge to make flexible choices by relying on two alternative strategies that do not require costly mental simulation at choice time. First, through offline replanning, models can be queried before the time of choice to generate possible scenarios and update the values of potential actions. Second, an abstracted predictive model, known as a Successor Representation, can be built and harnessed to enable simplified computation of long-run reward values of candidate actions, without requiring iterative simulation of multiple time steps. To assess whether children, adolescents, and adults aged 7 - 23 years similarly harness these learning strategies, we ran three experiments. In Experiments 1 and 2, we used a reward revaluation task in which we manipulated the opportunity for offline replanning during rest, and found that children flexibly updated their behavior by leveraging structured knowledge in an adult-like manner. Surprisingly, across age, rest did not mediate flexible replanning, raising the possibility that participants may have behaved adaptively by harnessing predictive representations online. In Experiment 3, we directly tested whether children use predictive representations. Here, we observed early-emerging use of the SR, providing a mechanistic account of how children use structured knowledge to guide choice without detailed model-based simulation.

From sequences of discrete events, humans build mental models of their world. Referred to as graph learning, the process produces a model encoding the graph of event-to-event transition probabilities. Recent evidence suggests that some networks are easier to learn than others, but the neural underpinnings of this effect remain unknown. Here we use fMRI to show that even over short timescales the network structure of a temporal sequence of stimuli determines the fidelity of event representations as well as the dimensionality of the space in which those representations are encoded: when the graph was modular as opposed to lattice-like, BOLD representations in visual areas better predicted trial identity and displayed higher intrinsic dimensionality. Broadly, our study shows that network context influences the strength of learned neural representations, motivating future work in the design, optimization, and adaptation of network contexts for distinct types of learning.

How do people model the world’s dynamics to guide mental simulation and evaluate choices? One prominent approach, the Successor Representation (SR), takes advantage of temporal abstraction of future states: by aggregating trajectory predictions over multiple timesteps, the brain can avoid the costs of iterative, multi-step mental simulation. Human behavior broadly shows signatures of such temporal abstraction, but finer-grained characterization of individuals’ strategies and their dynamic adjustment remains an open question. We developed a task to measure SR usage during dynamic, trial-by-trial learning. Using this approach, we find that participants exhibit a mix of SR and model-based learning strategies that varies across individuals. Further, by dynamically manipulating the task contingencies within-subject to favor or disfavor temporal abstraction, we observe evidence of resource-rational reliance on the SR, which decreases when future states are less predictable. Our work adds to a growing body of research showing that the brain arbitrates between approximate decision strategies. The current study extends these ideas from simple habits into usage of more sophisticated approximate predictive models, and demonstrates that individuals dynamically adapt these in response to the predictability of their environment.

Decisions in humans and other organisms depend, in part, on learning and using models that capture the statistical structure of the world, including the long-run expected outcomes of our actions. One prominent approach to forecasting such long-run outcomes is the successor representation (SR), which predicts future states aggregated over multiple timesteps. Although much behavioral and neural evidence suggests that people and animals use such a representation, it remains unknown how they acquire it. It has frequently been assumed to be learned by temporal difference bootstrapping (SR-TD(0)), but this assumption has largely not been empirically tested or compared to alternatives including eligibility traces (SR-TD( λ > 0)). Here we address this gap by leveraging trial-by-trial reaction times in graph sequence learning tasks, which are favorable for studying learning dynamics because the long horizons in these studies differentiate the transient update dynamics of different learning rules. We examined the behavior of SR-TD( λ ) on a probabilistic graph learning task alongside a number of alternatives, and found that behavior was best explained by a hybrid model which learned via SR-TD( λ ) alongside an additional predictive model of recency. The relatively large λ we estimate indicates a predominant role of eligibility trace mechanisms over the bootstrap-based chaining typically assumed. Our results provide insight into how humans learn predictive representations, and demonstrate that people simultaneously learn the SR alongside lower-order predictions. Author Summary: Our ability to plan intelligently requires predicting the state of the world multiple steps into the future. Enumerating future outcomes step-by-step, however, is slow and costly. Instead, research has shown that people rely on simplified models of the world that skip across multiple steps at once. How do we construct these simplified models? One promising idea is the successor representation (SR), which predicts future events via a simple and neurally plausible computation. The SR has been shown to explain a range of behavioral phenomena, but these studies have not identified which among many learning rules the brain uses to build the SR. Plausible mechanisms for learning associations over delays (called bootstrapping and eligibility traces) both converge to identical simplified world models, and thus existing studies on the SR, which focus on well trained behavior, are unable to distinguish between them. Here, we answer this question by examining behavior on a graph learning task, where stimulus-by-stimulus reaction times have been shown to reflect predictions over long temporal horizons. Through both model fitting and model-agnostic comparisons, we find that behavior is best explained by a learning rule heavily dependent on eligibility traces, in contrast to previous work which generally assumed an (untested) bootstrapping update rule.