Continuous Prediction of Pointing Targets With Motion and Eye-Tracking in Virtual Reality
Open Access
- 8 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 5933-5946
Abstract
We present a study on continuously predicting the direction to a pointing target in virtual environments using motion and eye-tracker data throughout the pointing process. We first collect time series data for user motion and eye-tracker in a cursorless, single-target pointing task. Results from analyzing fixation points from different sensors and observing velocity profiles over the course of pointing provide insights into optimally configuring features for predicting the target angles. Following this analysis, we train a recurrent neural network that feeds on sliding window inputs for continuously operating target direction prediction from start to finish. The input window contains historical data from past to current frames, capturing temporal changes in the feature data. By feeding on this input, our model can predict the direction of the target at any given time during pointing. Our findings demonstrate that incorporating eye-tracker data into the prediction model boosts the maximum achievable accuracy by 2.5 times when compared to baselines without eye-tracker data inputs. The results suggest that using features from both the eye-tracker and joint motion contributes to higher prediction performance, as well as faster stabilization of output values at the starting phase of pointing.Keywords
Funding Information
- NRF, South Korea (2022R1A4A5033689)
This publication has 26 references indexed in Scilit:
- Effects of physical walking on eyes-engaged target selection with ray-casting pointing in virtual realityVirtual Reality, 2022
- Modeling Pointing for 3D Target Selection in VR2021
- Unscripted Retargeting: Reach Prediction for Haptic Retargeting in Virtual Reality2021
- Outline Pursuits: Gaze-assisted Selection of Occluded Objects in Virtual Reality2020
- Head-Coupled Kinematic Template Matching: A Prediction Model for Ray Pointing in VR2020
- Eye&Head2019
- Towards accurate cursorless pointing: the effects of ocular dominance and handednessPersonal and Ubiquitous Computing, 2017
- Neuronal control of fixation and fixational eye movementsPhilosophical Transactions of the Royal Society B: Biological Sciences, 2017
- Modeling Distant Pointing for Compensating Systematic Displacements2015
- The impact of microsaccades on vision: towards a unified theory of saccadic functionNature Reviews Neuroscience, 2013