Characterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks
Preprint
Abstract
Theories for visually guided action account for online control in the presence of reliable sources of visual information, and predictive control to compensate for visuomotor delay and temporary occlusion. In this study, we characterize the temporal relationship between information integration window and prediction distance using computational models. Subjects were immersed in a simulated environment and attempted to catch virtual balls that were transiently "blanked" during flight. Recurrent neural networks were trained to reproduce subject's gaze and hand movements during blank. The models successfully predict gaze behavior within 3 degrees, and hand movements within 8.5 cm as far as 500 ms in time, with integration window as short as 27 ms. Furthermore, we quantified the contribution of each input source of information to motor output through an ablation study. The model is a proof of concept for prediction as a discrete mapping between information integrated over time and a temporally distant motor output.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Complexity matching in side-by-side walkingHuman Movement Science, 2017
- On-line and model-based approaches to the visual control of actionVision Research, 2015
- Long short-term memory recurrent neural network architectures for large scale acoustic modeling2014
- Memory and prediction in natural gaze controlPhilosophical Transactions of the Royal Society B: Biological Sciences, 2013
- Decoupling eye and hand movement control: Visual short-term memory influences reach planning more than saccade planningJournal of Vision, 2012
- Visuo-motor coordination and internal models for object interceptionExperimental Brain Research, 2009
- Strong anticipation: Sensitivity to long-range correlations in synchronization behaviorPhysica A: Statistical Mechanics and its Applications, 2008
- Visually timed action: time-out for ‘tau’?Trends in Cognitive Sciences, 1999
- The visual guidance of catchingExperimental Brain Research, 1993
- A Theory of Visual Control of Braking Based on Information about Time-to-CollisionPerception, 1976