Thomas H. B. FitzGerald

Probabilistic models of cognition typically assume that agents make inferences about current states by combining new sensory information with fixed beliefs about the past, an approach known as Bayesian filtering. This is computationally parsimonious, but, in general, leads to suboptimal beliefs about past states, since it ignores the fact that new observations typically contain information about the past as well as the present. This is disadvantageous both because knowledge of past states may be intrinsically valuable, and because it impairs learning about fixed or slowly changing parameters of the environment. For these reasons, in offline data analysis it is usual to infer on every set of states using the entire time series of observations, an approach known as (fixed-interval) Bayesian smoothing. Unfortunately, however, this is impractical for real agents, since it requires the maintenance and updating of beliefs about an ever-growing set of states. We propose an intermediate approach, finite retrospective inference (FRI), in which agents perform update beliefs about a limited number of past states (Formally, this represents online fixed-lag smoothing with a sliding window). This can be seen as a form of bounded rationality in which agents seek to optimize the accuracy of their beliefs subject to computational and other resource costs. We show through simulation that this approach has the capacity to significantly increase the accuracy of both inference and learning, using a simple variational scheme applied to both randomly generated Hidden Markov models (HMMs), and a specific application of the HMM, in the form of the widely used probabilistic reversal task. Our proposal thus constitutes a theoretical contribution to normative accounts of bounded rationality, which makes testable empirical predictions that can be explored in future work.

Probabilistic models of cognition typically assume that agents make inferences about current states by combining new sensory information with fixed beliefs about the past, an approach known as Bayesian filtering. This is computationally parsimonious, but, in general, leads to suboptimal beliefs about past states, since it ignores the fact that new observations typically contain information about the past as well as the present. This is disadvantageous both because knowledge of past states may be intrinsically valuable, and because it impairs learning about fixed or slowly changing parameters of the environment. For these reasons, in offline data analysis it is usual to infer on every set of states using the entire time series of observations, an approach known as (fixed-interval) Bayesian smoothing. Unfortunately, however, this is impractical for real agents, since it requires the maintenance and updating of beliefs about an ever-growing set of states. We propose an intermediate approach, finite retrospective inference (FRI), in which agents perform update beliefs about a limited number of past states. (Formally, this represents online fixed-lag smoothing with a sliding window.) This can be seen as a form of bounded rationality in which agents seek to optimise the accuracy of their beliefs subject to computational and other resource costs. We show through simulation that this approach has the capacity to significantly increase the accuracy of both inference and learning, using a simple variational scheme applied to both randomly generated Hidden Markov models (HMMs), and a specific application of the HMM, in the form of the widely used probabilistic reversal task. Our proposal thus constitutes a theoretical contribution to normative accounts of bounded rationality, which makes testable empirical predictions that can be explored in future work.

Determining the best model or models for a particular data set, a process known as Bayesian model comparison, is a critical part of probabilistic inference. Typically, this process assumes a fixed model-space (that is, a fixed set of candidate models). However, it is also possible to perform Bayesian inference over model-spaces themselves, thus determining which spaces provide the best explanation for observed data. Model-space inference (MSI) allows the effective exclusion of poorly performing models (a process analogous to Automatic Relevance Detection), and thus mitigates against the well-known phenomenon of model dilution, resulting in posterior probability estimates that are, on average, more accurate than those produced when using a fixed model-space. We focus on model comparison in the context of multiple independent data sets (as produced, for example, by multi-subject behavioural or neuroimaging studies), and cast our proposal as a development of random-effects Bayesian Model Selection, the current state-of-the-art in the field. We demonstrate the increased accuracy of MSI using simulated behavioural and neuroimaging data, as well as by assessing predictive performance in previously-acquired empirical data. Additionally, we explore other applications of MSI, including formal testing for a diversity of models within a population, and comparison of model-spaces between populations. Our approach thus provides an important new tool for model comparison.

Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. ‘Hidden state’ exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, ‘model parameter’ exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of ‘Bayes-optimal’ behaviour. Our findings provide a computational framework to understand how distinct levels of uncertainty induce different modes of information-gain in decision-making.

Our environment abounds in objects of different orientations. When we want to interact with these objects, e.g. by grasping, we can do so provided we accurately perceive their orientation. The orientation of an object can be characterized along three independent spatial axes: x (or slant), y (or tilt), and z (or depth). How does the human brain achieve the percept of object orientation, is it by processing each axis separately, or by processing them conjointly? Also, do different (combinations of) monocular and binocular depth cues affect our perception of depth differently? To answer these questions, we designed a behavioral task in which participants were asked to reproduce the orientation of a target array by adjusting the tilt and slant of a test array. On each trial, random combinations of target orientations on both X and Y were generated. Error magnitudes for each trial and for each axis were recorded. Over the course of three behavioral experiments, we coupled this task with an incremental number of depth cues, such that: i) only texture gradients were available; ii) texture gradients were coupled with line vergeance; iii) texture gradients, line vergeance and binocular disparity were used together. Our results appear to indicate a differential coupling strength between X and Y orientation perception according to which depth cues are available. In a follow-up fMRI study using the same behavioral procedure and texture gradients only, we presented not only trials where X and Y orientations were generated together, but also trials where only one of these axes was manipulated. The neuroimaging data have been analyzed through both effective and functional approaches, to further specify the brain areas involved in object orientation perception, as well as their coupling. Meeting abstract presented at VSS 2018

Normative models of human cognition often appeal to Bayesian filtering, which provides optimal online estimates of unknown or hidden states of the world, based on previous observations. However, in many cases it is necessary to optimise beliefs about sequences of states rather than just the current state. Importantly, Bayesian filtering and sequential inference strategies make different predictions about beliefs and subsequent choices, rendering them behaviourally dissociable. Taking data from a probabilistic reversal task we show that subjects’ choices provide strong evidence that they are representing short sequences of states. Between-subject measures of this implicit sequential inference strategy had a neurobiological underpinning and correlated with grey matter density in prefrontal and parietal cortex, as well as the hippocampus. Our findings provide, to our knowledge, the first evidence for sequential inference in human cognition, and by exploiting between-subject variation in this measure we provide pointers to its neuronal substrates. When studying human cognition, it is often assumed that agents form and update beliefs only about the current state of the world, an approach known as Bayesian filtering. However, in many situations there are advantages to making inferences about the most likely sequence of states that have occurred, which involves simultaneously updating beliefs about the present and the past, based on incoming information. Currently, very little is known about whether humans adopt such sequential inference strategies, and if they do, about the neuronal mechanisms involved. We addressed this by applying computational modelling to data collected during a probabilistic reversal task. At a group level, subjects’ behaviour showed clear evidence of sequential inference, and between-subject differences in the strategies adopted were reflected in variations in brain structure in the prefrontal and parietal cortices, as well as the hippocampus. Our results provide new insight into the strategies employed in human cognition, as well as the neuronal substrates of sequential inference.

This article describes a process theory based on active inference and belief propagation. Starting from the premise that all neuronal processing (and action selection) can be explained by maximizing Bayesian model evidence—or minimizing variational free energy—we ask whether neuronal responses can be described as a gradient descent on variational free energy. Using a standard (Markov decision process) generative model, we derive the neuronal dynamics implicit in this description and reproduce a remarkable range of well-characterized neuronal phenomena. These include repetition suppression, mismatch negativity, violation responses, place-cell activity, phase precession, theta sequences, theta-gamma coupling, evidence accumulation, race-to-bound dynamics, and transfer of dopamine responses. Furthermore, the (approximately Bayes’ optimal) behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton’s principle of least action.