Jeff B. Pelz
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jeff B. Pelz has a total of 135 co-authors. Here are the top co-authors by number of shared publications:
Pengcheng Shi
Golisano College of Computing and Information Science, Rochester Institute of Technology, 1 Lomb Memorial Drive Rochester, NY 14623, USA
8 shared publications- Cecilia Ovesdotter Alm
Rochester Institute of Technology, Rochester, NY, USA
7 shared publications Jason S. Babcock
6 shared publicationsPreethi Vaidyanathan
6 shared publicationsAnne Haake
Rochester Institute of Technology
5 shared publicationsAnne R. Haake
Golisano College of Computing and Information Science, Rochester Institute of Technology, 1 Lomb Memorial Drive Rochester, NY 14623, USA
5 shared publications
Latest Publications
- Open Access
- 25 May 2021
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Enhancing the precision of remote eye-tracking using iris velocity estimation
Most of the previous work on eye-tracking has focused on positional information of the eye features. Recent advances in camera technology such as high-resolution and event cameras allow consideration of the velocity estimate for eye tracking. Some previous work on velocity-based estimates has demonstrated high-precision gaze estimation by tracking the motion of iris features on high-resolution images rather than by exploiting pupil edges. While these methods provide high precision, the bottleneck for velocity-based methods are temporal drift and the inability to track across blinks. In this work, we present a new theoretical methodology (πt) to address these issues by optimally combining low-temporal frequency components of the pupil edges with the high-temporal frequency components from the iris textures. We show improved precision with this method while fixating a series of small targets and following a smoothly moving target. Further, we demonstrate the capability to reliably identify microsaccades between targets separated by 0.2°.
- 25 May 2021
- Published by Association for Computing Machinery (ACM)
- p. 1-7
Open AccessEnhancing the precision of remote eye-tracking using iris velocity estimation
Most of the previous work on eye-tracking has focused on positional information of the eye features. Recent advances in camera technology such as high-resolution and event cameras allow consideration of the velocity estimate for eye tracking. Some previous work on velocity-based estimates has demonstrated high-precision gaze estimation by tracking the motion of iris features on high-resolution images rather than by exploiting pupil edges. While these methods provide high precision, the bottleneck for velocity-based methods are temporal drift and the inability to track across blinks. In this work, we present a new theoretical methodology (πt) to address these issues by optimally combining low-temporal frequency components of the pupil edges with the high-temporal frequency components from the iris textures. We show improved precision with this method while fixating a series of small targets and following a smoothly moving target. Further, we demonstrate the capability to reliably identify microsaccades between targets separated by 0.2°.
- Open Access
- 1 January 2021
- Published by Elsevier in Procedia Computer Science
- Vol. 192, 2952-2961
GazeEnViz4D: 4-D Gaze-in-Environment Visualization Pipeline
- 1 January 2021
- Published by Elsevier in Procedia Computer Science
- Vol. 192, 2952-2961
Open AccessGazeEnViz4D: 4-D Gaze-in-Environment Visualization Pipeline
- Open Access
- 1 July 2020
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 20 (7) , 13
Computational framework for fusing eye movements and spoken narratives for image annotation
Despite many recent advances in the field of computer vision, there remains a disconnect between how computers process images and how humans understand them. To begin to bridge this gap, we propose a framework that integrates human-elicited gaze and spoken language to label perceptually important regions in an image. Our work relies on the notion that gaze and spoken narratives can jointly model how humans inspect and analyze images. Using an unsupervised bitext alignment algorithm originally developed for machine translation, we create meaningful mappings between participants’ eye movements over an image and their spoken descriptions of that image. The resulting multimodal alignments are then used to annotate image regions with linguistic labels. The accuracy of these labels exceeds that of baseline alignments obtained using purely temporal correspondence between fixations and words. We also find differences in system performances when identifying image regions using clustering methods that rely on gaze information rather than image features. The alignments produced by our framework can be used to create a database of low-level image features and high-level semantic annotations corresponding to perceptually important image regions. The framework can potentially be applied to any multimodal data stream and to any visual domain. To this end, we provide the research community with access to the computational framework.
- 1 July 2020
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 20 (7) , 13
Open AccessComputational framework for fusing eye movements and spoken narratives for image annotation
Despite many recent advances in the field of computer vision, there remains a disconnect between how computers process images and how humans understand them. To begin to bridge this gap, we propose a framework that integrates human-elicited gaze and spoken language to label perceptually important regions in an image. Our work relies on the notion that gaze and spoken narratives can jointly model how humans inspect and analyze images. Using an unsupervised bitext alignment algorithm originally developed for machine translation, we create meaningful mappings between participants’ eye movements over an image and their spoken descriptions of that image. The resulting multimodal alignments are then used to annotate image regions with linguistic labels. The accuracy of these labels exceeds that of baseline alignments obtained using purely temporal correspondence between fixations and words. We also find differences in system performances when identifying image regions using clustering methods that rely on gaze information rather than image features. The alignments produced by our framework can be used to create a database of low-level image features and high-level semantic annotations corresponding to perceptually important image regions. The framework can potentially be applied to any multimodal data stream and to any visual domain. To this end, we provide the research community with access to the computational framework.
- Open Access
Characterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks
Theories for visually guided action account for online control in the presence of reliable sources of visual information, and predictive control to compensate for visuomotor delay and temporary occlusion. In this study, we characterize the temporal relationship between information integration window and prediction distance using computational models. Subjects were immersed in a simulated environment and attempted to catch virtual balls that were transiently "blanked" during flight. Recurrent neural networks were trained to reproduce subject's gaze and hand movements during blank. The models successfully predict gaze behavior within 3 degrees, and hand movements within 8.5 cm as far as 500 ms in time, with integration window as short as 27 ms. Furthermore, we quantified the contribution of each input source of information to motor output through an ablation study. The model is a proof of concept for prediction as a discrete mapping between information integrated over time and a temporally distant motor output.
Open AccessCharacterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks
Theories for visually guided action account for online control in the presence of reliable sources of visual information, and predictive control to compensate for visuomotor delay and temporary occlusion. In this study, we characterize the temporal relationship between information integration window and prediction distance using computational models. Subjects were immersed in a simulated environment and attempted to catch virtual balls that were transiently "blanked" during flight. Recurrent neural networks were trained to reproduce subject's gaze and hand movements during blank. The models successfully predict gaze behavior within 3 degrees, and hand movements within 8.5 cm as far as 500 ms in time, with integration window as short as 27 ms. Furthermore, we quantified the contribution of each input source of information to motor output through an ablation study. The model is a proof of concept for prediction as a discrete mapping between information integrated over time and a temporally distant motor output.
- 28 January 2018
- Published by Society for Imaging Science & Technology in Electronic Imaging
- Vol. 30 (17) , 163-1
Auto-simulator Preparation for Research into Assessing the Correlation Between Human Driving Behaviors and Fixation Patterns
Autonomous driving has the potential to positively impact the daily life of humans. Techniques such as imaging processing, computer vision, and remote sensing have been highly involved in creating reliable and secure robotic cars. Conversely, the interaction between human perception and autonomous driving has not been deeply explored. Therefore, the analysis of human perception during the cognitive driving task, while making critical driving decisions, may provide great benefits for the study of autonomous driving. To achieve such an analysis, eye movement data of human drivers was collected with a mobile eye-tracker while driving in a automotive simulator built around an actual physical car, that mimics a realistic driving experience. Initial experiments have been performed to investigate the potential correlation between the driving behaviors and fixation patterns of the human driver.
- 28 January 2018
- Published by Society for Imaging Science & Technology in Electronic Imaging
- Vol. 30 (17) , 163-1
Auto-simulator Preparation for Research into Assessing the Correlation Between Human Driving Behaviors and Fixation Patterns
Autonomous driving has the potential to positively impact the daily life of humans. Techniques such as imaging processing, computer vision, and remote sensing have been highly involved in creating reliable and secure robotic cars. Conversely, the interaction between human perception and autonomous driving has not been deeply explored. Therefore, the analysis of human perception during the cognitive driving task, while making critical driving decisions, may provide great benefits for the study of autonomous driving. To achieve such an analysis, eye movement data of human drivers was collected with a mobile eye-tracker while driving in a automotive simulator built around an actual physical car, that mimics a realistic driving experience. Initial experiments have been performed to investigate the potential correlation between the driving behaviors and fixation patterns of the human driver.
- Open Access
- 1 January 2018
- Published by Association for Computational Linguistics (ACL)
SNAG: Spoken Narratives and Gaze Dataset
- 1 January 2018
- Published by Association for Computational Linguistics (ACL)
Open AccessSNAG: Spoken Narratives and Gaze Dataset
- 1 November 2017
- Published by Elsevier in Forensic Science International
- Vol. 280, 64-80
Eye tracking to evaluate evidence recognition in crime scene investigations
- 1 November 2017
- Published by Elsevier in Forensic Science International
- Vol. 280, 64-80
Eye tracking to evaluate evidence recognition in crime scene investigations
- 21 September 2016
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 151, 138-152
Modeling eye movement patterns to characterize perceptual skill in image-based diagnostic reasoning processes
- 21 September 2016
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 151, 138-152
Modeling eye movement patterns to characterize perceptual skill in image-based diagnostic reasoning processes
- Open Access
- 22 July 2016
- Published by Association for Computing Machinery (ACM)
- p. 15-18
Binocular eye tracking calibration during a virtual ball catching task using head mounted display
When tracking the eye movements of an active observer, the quality of the tracking data is continuously affected by physical shifts of the eye-tracker on an observers head. This is especially true for eye-trackers integrated within virtual-reality (VR) helmets. These configurations modify the weight and inertia distribution well beyond that of the eye-tracker alone. Despite the continuous nature of this degradation, it is common practice for calibration procedures to establish eye-to-screen mappings, fixed over the time-course of an experiment. Even with periodic recalibration, data quality can quickly suffer due to head motion. Here, we present a novel post-hoc calibration method that allows for continuous temporal interpolation between discrete calibration events. Analysis focuses on the comparison of fixed vs. continuous calibration schemes and their effects upon the quality of a binocular gaze data to virtual targets, especially with respect to depth. Calibration results were applied to binocular eye tracking data from a VR ball catching task and improved the tracking accuracy especially in the dynamic case.
- 22 July 2016
- Published by Association for Computing Machinery (ACM)
- p. 15-18
Open AccessBinocular eye tracking calibration during a virtual ball catching task using head mounted display
When tracking the eye movements of an active observer, the quality of the tracking data is continuously affected by physical shifts of the eye-tracker on an observers head. This is especially true for eye-trackers integrated within virtual-reality (VR) helmets. These configurations modify the weight and inertia distribution well beyond that of the eye-tracker alone. Despite the continuous nature of this degradation, it is common practice for calibration procedures to establish eye-to-screen mappings, fixed over the time-course of an experiment. Even with periodic recalibration, data quality can quickly suffer due to head motion. Here, we present a novel post-hoc calibration method that allows for continuous temporal interpolation between discrete calibration events. Analysis focuses on the comparison of fixed vs. continuous calibration schemes and their effects upon the quality of a binocular gaze data to virtual targets, especially with respect to depth. Calibration results were applied to binocular eye tracking data from a VR ball catching task and improved the tracking accuracy especially in the dynamic case.
- 6 July 2016
- Published by Springer Nature in Behavior Research Methods
- Vol. 49 (3) , 947-959
A study of artificial eyes for the measurement of precision in eye-trackers
The precision of an eye-tracker is critical to the correct identification of eye movements and their properties. To measure a system's precision, artificial eyes (AEs) are often used, to exclude eye movements influencing the measurements. A possible issue, however, is that it is virtually impossible to construct AEs with sufficient complexity to fully represent the human eye. To examine the consequences of this limitation, we tested currently used AEs from three manufacturers of eye-trackers and compared them to a more complex model, using 12 commercial eye-trackers. Because precision can be measured in various ways, we compared different metrics in the spatial domain and analyzed the power-spectral densities in the frequency domain. To assess how precision measurements compare in artificial and human eyes, we also measured precision using human recordings on the same eye-trackers. Our results show that the modified eye model presented can cope with all eye-trackers tested and acts as a promising candidate for further development of a set of AEs with varying pupil size and pupil-iris contrast. The spectral analysis of both the AE and human data revealed that human eye data have different frequencies that likely reflect the physiological characteristics of human eye movements. We also report the effects of sample selection methods for precision calculations. This study is part of the EMRA/COGAIN Eye Data Quality Standardization Project.
- 6 July 2016
- Published by Springer Nature in Behavior Research Methods
- Vol. 49 (3) , 947-959
A study of artificial eyes for the measurement of precision in eye-trackers
The precision of an eye-tracker is critical to the correct identification of eye movements and their properties. To measure a system's precision, artificial eyes (AEs) are often used, to exclude eye movements influencing the measurements. A possible issue, however, is that it is virtually impossible to construct AEs with sufficient complexity to fully represent the human eye. To examine the consequences of this limitation, we tested currently used AEs from three manufacturers of eye-trackers and compared them to a more complex model, using 12 commercial eye-trackers. Because precision can be measured in various ways, we compared different metrics in the spatial domain and analyzed the power-spectral densities in the frequency domain. To assess how precision measurements compare in artificial and human eyes, we also measured precision using human recordings on the same eye-trackers. Our results show that the modified eye model presented can cope with all eye-trackers tested and acts as a promising candidate for further development of a set of AEs with varying pupil size and pupil-iris contrast. The spectral analysis of both the AE and human data revealed that human eye data have different frequencies that likely reflect the physiological characteristics of human eye movements. We also report the effects of sample selection methods for precision calculations. This study is part of the EMRA/COGAIN Eye Data Quality Standardization Project.