Laurence R. Harris

The movement of moving objects seen during self-motion is incorrectly perceived as a result of errors in parsing the visual motion arising from the retina and from information about the movement of the head and eyes in space ¹ . The perception of motion of auditory targets may be even more challenging as previous research suggests that the auditory system lacks dedicated low-level speed processing mechanisms. This study investigates how the perception of a moving auditory sound source is influenced by concurrent head movements, drawing comparisons with established visual motion perception findings and illusions, such as the Filehne Illusion. Participants were asked to move their heads at a fixed speed, either horizontally (left to right or right to left) or vertically (up to down or down to up), while indicating the perceived direction of a moving sound source by using a joystick. A physical sound source was moved silently at a constant speed in fronto-parallel 2D space using a 3 × 3×3m cable robot. Results showed that head movements created a perceptual bias of the perceived direction of the moving sound source in the opposite direction to the head movement. Responses were less precise during horizontal than vertical head movement, whereas overall directional accuracy did not differ reliably between movement planes. The magnitude of the directional error also depended on the direction of head movement, consistent with the different cues available for horizontal and vertical localization (binaural vs. spectral cues). These findings suggest that similarly to how visual motion perception is distorted by extraretinal signals and the movement of the eyes, so too is auditory motion perception influenced by movement of the ears, revealing errors in the integration of proprioceptive, vestibular and auditory cues that distort the perceived direction of a moving sound source.

In this viewpoint, Laurence Harris and Marc Ernst reflect on their careers studying how the brain combines information from multiple sensory systems — vision, hearing, touch, and the vestibular sense — to perceive the world and guide action. They trace the field’s evolution from early psychophysical and neurophysiological work to the introduction of Bayesian models, which reframed multisensory integration as a problem of statistical inference under uncertainty and provided a principled link between behavioural findings and normative theory. Both authors highlight open challenges: bridging computational, algorithmic, and neural levels of explanation; understanding how the brain represents and approximates complex, context-dependent probabilistic information; and characterising the naturalistic sensory signals generated during active behaviours such as walking, grasping, and navigating. They discuss the growing role of virtual reality in enabling more ecologically valid experiments while preserving experimental control and note that progress has been uneven across modalities — most advanced in vision, less so in audition and haptics. The authors also address open-science practices, the value and limits of replication, and the anticipated role of artificial intelligence as a tool that accelerates research workflows rather than replaces scientific reasoning. They close with reflections on mentorship and curiosity as drivers of scientific careers, and a shared conviction that no sense operates in isolation — multisensory integration is fundamental to perception and action and remains one of the most promising frontiers in cognitive science.

As observers move through the world, optic flow provides critical information about self-motion, including direction (heading). Over the years, studies of heading perception have reported conflicting biases: some toward straight ahead (centripetal), others away from it (centrifugal). We hypothesize that these discrepancies arise from differences in the reference frame used to report heading. We tested this hypothesis using a unique “Edgeless Graphics Geometry” (EGG) display with an ultra-wide (~ 224°) field of view. Participants viewed brief optic flow stimuli simulating self-motion across a ground plane. They reported heading either egocentrically (relative to themselves) or allocentrically (relative to external space). In the same observers and despite identical response ranges (± 110°), egocentric reports exhibited a robust bias towards straight ahead, whereas allocentric reports showed a pronounced bias away from straight ahead. These differences suggest a reference-frame–dependent coordinate transformation of visual self-motion signals dependent on the task in hand. To identify neural correlates of this transformation, we combined a two alternative forced choice heading task with non-invasive scalp EEG. Following stimulation, participants were cued to respond in either an egocentric or an allocentric reference frame. Allocentric responses were associated with a stronger post-cue increase in theta-band power than egocentric responses, suggesting differential retrosplenial and (para-)hippocampal networks’ involvement in these two types of response preparation. Finally, a Bayesian causal inference model captured both these behavioral and neural findings, providing a principled account of how reference-frame transformations generate opposing biases in heading perception, thereby resolving a long-standing issue in vision science.

Accurately perceiving the duration of stimuli is important for many human behaviours, especially those involving movement such as when catching a ball. However, perceived duration is reportedly different depending on which sense is being monitored, with auditory stimuli often judged as longer than a matched visual stimulus. Recent experiments have implicated gravity in duration perception, but rigorous psychophysical methods have been lacking. Here, we developed a novel psychometric method for measuring perceived duration and manipulated the reliability of gravity by altering posture and by disruptive galvanic vestibular stimulation (dGVS). We assessed people’s judgements of 1 s using visual (an LED), auditory (200 Hz at 56 dB played through headphones) or tactile (200 Hz played through tactors mounted on the backs of the hands) stimuli. Participants judged whether each stimulus, the duration of which was controlled by a QUick Estimation by Sequential Testing (QUEST) staircase, was longer or shorter than their internal estimate of 1 s in two postures: supine and standing. We also compared visual duration judgements in the presence of dGVS with those collected during sham stimulation to the neck. The perceived durations of auditory and tactile events were not affected by changing posture, but the perceived duration of a visual second was significantly longer when supine compared to when upright (by 45 ms) or during dGVS compared to during sham stimulation (by 66 ms). The duration of a light or tactile stimulus perceived as having a duration of 1 s was longer than the time needed for an auditory event to be perceived as being on for 1 s. We conclude that disrupting the vestibular system by lying supine or by electrical stimulation influences visual duration perception, but the perceived duration of auditory and tactile stimuli remains unaffected.

During sideways movement of an observer, optic flow parsing – in which an object’s speed in the world is extracted from all the other visual movement present in the scene, self-generated and otherwise – has been shown to be incomplete, leading to biases in speed perception, particularly when object and observer are moving in opposite directions. Here, we assess how judgements about the speed of objects moving in depth (judged relative to the world) towards or away from an observer (6 m/s) are affected by simultaneous movement of the observer either in the same or opposite direction as the object. In a virtual reality display, participants (n = 25) viewed a sphere simulated as moving in a corridor either while they were stationary or during visually simulated self-motion in the same or opposite direction as the object. They judged the sphere’s movement relative to the world by comparing its motion to a probe sphere that travelled laterally across the corridor in front of them. In a second experiment (n = 28) participants performed the same task but during faster self-motion (10 m/s). The second cohort also judged the direction in which the object was perceived to move during the same combinations of self and object speeds. Object speed was overestimated when the object travelled in the direction opposite to the observer compared to how object’s motion was judged when the observer was stationary. However, object speed was also overestimated during self-motion in the same direction as the object where participants were also much more likely to misjudge the direction of motion of the object. Precision of judgements was lower when self-motion was simulated than it was for stationary observers. A simple arithmetic model of flow parsing fails to capture these results satisfactorily, suggesting that different mechanisms may be at play when the observer travels in the same direction as a moving object and is vulnerable to misperceiving its direction of travel.

Continuous Psychophysics, which couples a continuous stimulus with a continuous response, is a promising tool to break out of the confines of traditional designs based on discrete trials. In this pre-registered study, we explore to what extent this paradigm is useful in the study of multisensory integration. We expand on Tonelli et al.’s (2025) seminal study by additionally examining the role of eye-movements, using a Kalman filter to estimate the sensory noise underlying behavioral tracking parameters and employing a virtual reality set-up. We immersed two cohorts of participants (n = 30 each) in a virtual meadow environment and asked them to continuously track a drone (Experiment 1) or a swarm of flies (Experiment 2) with a controller, while simultaneously recording their eye movements. We manipulated the reliability of visual cues using four levels of fog (from a completely clear view to impenetrable fog where no visual cues to the target’s position were available) as well as the presence of sound cues emitted from the object (sound present/absent). The maximum correlation between stimulus and response was higher when sound was present in some conditions, particularly when visual uncertainty was high, while the tracking delay remained unaffected across all fog levels. Using a Kalman filter to estimate the underlying sensory noise, we found strong evidence that sensory noise was lower when sound was present than when sound was absent both for manual and for ocular tracking, particularly for those conditions with higher visual uncertainty. In exploratory analyses, we further show strong correlations between manual and ocular tracking in all measures (maximum correlation, tracking delay, sensory precision). However, when isolating the multisensory advantage, these correlations all but disappeared for maximum correlation and tracking delay, while remaining substantial for sensory precision. Similarly, behavioral tracking correlated generally strongly with underlying sensory noise, but much less so when it came to the advantage conferred by added sound cues. Our results show that continuous psychophysics is well-suited for the study of multisensory integration, particularly when a Kalman filter analysis is used to estimate sensory uncertainty from behavioral data.

Previous research has investigated variations in the effectiveness of audio-visual (AV) integration dependent on location relative to the observer, with inconsistent results. Here, we examine AV interactions in the 3D space around an observer and address six factors that may contribute to these inconsistencies. Using a redundant-targets-effect paradigm in virtual reality, we conducted speeded detection and localization tasks to randomly intermixed auditory, visual and audio-visual stimuli presented in near or far, left or right regions of space around an observer. We varied stimulus characteristics to control for distance-related magnitude variations, examined static and looming stimuli, and analysed response times, multisensory benefits, and race model violations across conditions. Our findings reveal location-related effects on AV integration for looming but not stationary stimuli. Specifically, we observed near-space enhancement for AV looming stimuli for participants’ sensory-motor responses and a left/near space enhancement for the multisensory benefit. Our method of intermixing stimulus locations and magnitude adjustments to control for inverse effectiveness was critical for demonstrating these effects. Task goals modified outcomes in complex ways. These results provide new insights into AV integration in 3D space, extend previous findings and highlight the importance and limitations of methodological factors.

Fear of falling in older adults is generally studied as a chronic condition, yet spikes in acute fear of falling remain underexplored, despite previous research showing that they often precipitate falls. This work introduces a multi-modal sensor-based framework for detecting theoretically defined 'potential fear of falling' by combining gaze elevation and heart rate signals captured from wearable eye tracking and wrist devices in older adults living in the community. We trained five conventional classifiers (logistic regression, KNN, random forest, XGBoost, CatBoost), optimized for minority class F1, and combined two ensembles: (1) random forest + CatBoost + KNN and (2) random forest + logistic regression + KNN. We also applied spectrogram-based transfer learning by fine-tuning the pre-trained VGG16 and ResNet50 models on accelerometer data. In the individual-classifier analysis, XGBoost, KNN, and random forest achieved ROC AUC = 0.99 and minority-class F1 of 0.93, 0.90, and 0.85, respectively. The ensemble models performed better than individual classifiers on multi-modal and accelerometer-only inputs, though overall performance remained modest without multi-modal signals in the latter case (minority-class F1 = 0.39). Transfer models outperformed ensembles. These results demonstrate that ensemble and spectrogram-based transfer learning models provide robust, high-sensitivity detection of potential acute fear of falling in multi-modal signals. This work lays the foundation for future studies to explore acute fear of falling biomarkers in larger cohorts and paves the way for personalized fall prevention interventions in everyday settings.

For many years the accepted wisdom in vision science has been that humans tend to underestimate the speed of an object when they pursue it with their gaze in comparison to when they fixate a spot in the visual scene and the object moves across the retina, an effect that is referred to as the Aubert-Fleischl phenomenon. However, experiments on the Aubert-Fleischl effect have generally employed a target moving in front of a blank background. For objects moving in the real world, relative motion between the object and the background provides additional cues, potentially allowing the visual system to compensate for the Aubert-Fleischl phenomenon. To test this hypothesis, we asked 50 participants to compare the speed of a single sphere to the speed of a sphere-cloud in a two-interval forced-choice task while they either followed the single sphere with their gaze or kept their eyes on a fixation cross. Stimuli were presented in virtual reality and the sphere’s movement could occur either in a completely black environment with no relative motion cues present, or against a visible, textured background behind the moving sphere. The probe sphere-cloud always moved in front of the same background as the target. In a large data set (n = 50), we found no evidence for an Aubert-Fleischl effect, i.e., no evidence for an underestimation of speed during pursuit relative to fixation, in either of the environments. Our results challenge the prevailing notion that object speed is underestimated when the object is pursued versus when fixation is maintained and highlights the necessity to take into account characteristics of the visual scene.