Tom Hartley

Processing pathways between sensory and default mode network (DMN) regions support recognition, navigation, and memory but their organisation is not well understood. We show that functional subdivisions of visual cortex and DMN sit at opposing ends of parallel streams of information processing that support visually-mediated semantic and spatial cognition, providing convergent evidence from univariate and multivariate task responses, intrinsic functional and structural connectivity. Participants learned virtual environments consisting of buildings populated with objects, drawn from either a single semantic category or multiple categories. Later, they made semantic and spatial context decisions about these objects and buildings during functional magnetic resonance imaging. A lateral ventral occipital to frontotemporal DMN pathway was primarily engaged by semantic judgements, while a medial visual to medial temporal DMN pathway supported spatial context judgements. These pathways had distinctive locations in functional connectivity space: the semantic pathway was both further from unimodal systems and more balanced between visual and auditory-motor regions compared with the spatial pathway. When semantic and spatial context information could be integrated (in buildings containing objects from a single category), regions at the intersection of these pathways responded, suggesting that parallel processing streams interact at multiple levels of the cortical hierarchy to produce coherent memory-guided cognition.

Processing pathways between sensory and default mode network (DMN) regions support recognition, navigation, and memory but their organisation is not well understood. We show that functional subdivisions of visual cortex and DMN sit at opposing ends of parallel streams of information processing that support visually-mediated semantic and spatial cognition, providing convergent evidence from univariate and multivariate task responses, intrinsic functional and structural connectivity. Participants learned virtual environments consisting of buildings populated with objects, drawn from either a single semantic category or multiple categories. Later, they made semantic and spatial context decisions about these objects and buildings during functional magnetic resonance imaging. A lateral ventral occipital to frontotemporal DMN pathway was primarily engaged by semantic judgements, while a medial visual to medial temporal DMN pathway supported spatial context judgements. These pathways had distinctive locations in functional connectivity space: the semantic pathway was both further from unimodal systems and more balanced between visual and auditory-motor regions compared with the spatial pathway. When semantic and spatial context information could be integrated (in buildings containing objects from a single category), regions at the intersection of these pathways responded, suggesting that parallel processing streams interact at multiple levels of the cortical hierarchy to produce coherent memory-guided cognition.

The brain encodes spatial information in at least two distinct ways: egocentric representations encode locations relative to the observer, whereas allocentric representations encode locations relative to the environment. Both inform spatial memory, but the extent to which they influence behaviour varies depending on the task. In two preregistered experiments we used a psychophysical approach to measure the precision of spatial memory while varying ego- and allocentric task demands. Participants attempted to detect the changed location of one of four objects when seen from a new viewpoint. In both experiments (one using a Same/Different task, the other a 2AFC task), psychophysically-determined change detection thresholds showed a monotonic but non-linear increase as the degree of viewpoint change increased. Our findings were consistent with a preregistered model which shows how, governed by distinct ego- and allocentric contributions, the precision of spatial memory changes lawfully as a function of viewpoint shift.

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low‐level or high‐level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data‐driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi‐dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high‐level, category‐selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

The binding of features into perceptual wholes is a well-established phenomenon, which has previously only been studied in the context of early vision and low-level features, such as color or proximity. We hypothesized that a similar binding process, based on higher level information, could bind people into interacting groups, facilitating faster processing and enhanced memory of social situations. To investigate this possibility we used 3 experimental approaches to explore grouping effects in displays involving interacting people. First, using a visual search task we demonstrate more rapid processing for interacting (vs. noninteracting) pairs in an odd-quadrant paradigm (Experiments 1a and 1b). Second, using a spatial judgment task, we show that interacting individuals are remembered as physically closer than are noninteracting individuals (Experiments 2a and 2b). Finally, we show that memory retention of group-relevant and irrelevant features are enhanced when recalling interacting partners in a surprise memory task (Experiments 3a and 3b). Each of these results is consistent with the social binding hypothesis, and alternative explanations based on low level perceptual features and attentional effects are ruled out. We conclude that automatic midlevel grouping processes bind individuals into groups on the basis of their perceived interaction. Such social binding could provide the basis for more sophisticated social processing. Identifying the automatic encoding of social interactions in visual search, distortions of spatial working memory, and facilitated retrieval of object properties from longer-term memory, opens new approaches to studying social cognition with possible practical applications. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Virtual reality presents a new set of challenges and opportunities for both engineers and neuroscientists. Here we provide an overview of a programme designed by a group of psychologists, neuroscientists and VR specialists to address some of the most outstanding issues in the field ranging from the very low-level (for example, how the brain processed motion-in-depth signals generated by stereoscopic display devices) to the very high level (how virtual environments can lead to a sense of immersion and emotional engagement). We present data from psychophysical, electrophysiological and neuroimaging experiments and explain how different research methodologies can be applied to different problems in the field of VR/AR. We end by describing an open-source, extensible software package for studying issues in VR that can interface to common laboratory measurement equipment and discussing future directions and challenges facing the neuroscience and VR engineering communities.

The capacity of serially ordered auditory–verbal short-term memory (AVSTM) is sensitive to the timing of the material to be stored, and both temporal processing and AVSTM capacity are implicated in the development of language. We developed a novel “rehearsal-probe” task to investigate the relationship between temporal precision and the capacity to remember serial order. Participants listened to a sub-span sequence of spoken digits and silently rehearsed the items and their timing during an unfilled retention interval. After an unpredictable delay, a tone prompted report of the item being rehearsed at that moment. An initial experiment showed cyclic distributions of item responses over time, with peaks preserving serial order and broad, overlapping tails. The spread of the response distributions increased with additional memory load and correlated negatively with participants’ auditory digit spans. A second study replicated the negative correlation and demonstrated its specificity to AVSTM by controlling for differences in visuo-spatial STM and nonverbal IQ. The results are consistent with the idea that a common resource underpins both the temporal precision and capacity of AVSTM. The rehearsal-probe task may provide a valuable tool for investigating links between temporal processing and AVSTM capacity in the context of speech and language abilities.

The neural representation of objects in the ventral visual pathway has been linked to high-level properties of the stimulus, such as semantic or categorical information. However, the extent to which patterns of neural response in these regions reflect more basic underlying principles is unclear. One problem is that existing studies generally employ stimulus conditions chosen by the experimenter, potentially obscuring the contribution of more basic stimulus dimensions. To address this issue, we used a data-driven analysis to describe a large database of objects in terms of their visual properties (spatial frequency, orientation, location). Clustering algorithms were then used to select images from distinct regions of this feature space. Images in each cluster did not clearly correspond to typical object categories. Nevertheless, they elicited distinct patterns of response in the ventral stream. Moreover, the similarity of the neural response across different clusters could be predicted by the similarity in image properties, but not by the similarity in semantic properties. These findings provide an image-based explanation for the emergence of higher-level representations of objects in the ventral visual pathway. Meeting abstract presented at VSS 2017