JPJeff B. Pelz

Jeff B. Pelz

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jeff B. Pelz has a total of 135 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Enhancing the precision of remote eye-tracking using iris velocity estimation

    by Aayush Kumar Chaudhary, Jeff B Pelz

    Most of the previous work on eye-tracking has focused on positional information of the eye features. Recent advances in camera technology such as high-resolution and event cameras allow consideration of the velocity estimate for eye tracking. Some previous work on velocity-based estimates has demonstrated high-precision gaze estimation by tracking the motion of iris features on high-resolution images rather than by exploiting pupil edges. While these methods provide high precision, the bottleneck for velocity-based methods are temporal drift and the inability to track across blinks. In this work, we present a new theoretical methodology (πt) to address these issues by optimally combining low-temporal frequency components of the pupil edges with the high-temporal frequency components from the iris textures. We show improved precision with this method while fixating a series of small targets and following a smoothly moving target. Further, we demonstrate the capability to reliably identify microsaccades between targets separated by 0.2°.


  • Open Access

    GazeEnViz4D: 4-D Gaze-in-Environment Visualization Pipeline

    by Anjali Kishore Jogeshwar, Jeff B. Pelz


  • Open Access

    Computational framework for fusing eye movements and spoken narratives for image annotation

    by Preethi Vaidyanathan, Emily Prud’hommeaux, Cecilia O. Alm, Jeff B. Pelz

    Despite many recent advances in the field of computer vision, there remains a disconnect between how computers process images and how humans understand them. To begin to bridge this gap, we propose a framework that integrates human-elicited gaze and spoken language to label perceptually important regions in an image. Our work relies on the notion that gaze and spoken narratives can jointly model how humans inspect and analyze images. Using an unsupervised bitext alignment algorithm originally developed for machine translation, we create meaningful mappings between participants’ eye movements over an image and their spoken descriptions of that image. The resulting multimodal alignments are then used to annotate image regions with linguistic labels. The accuracy of these labels exceeds that of baseline alignments obtained using purely temporal correspondence between fixations and words. We also find differences in system performances when identifying image regions using clustering methods that rely on gaze information rather than image features. The alignments produced by our framework can be used to create a database of low-level image features and high-level semantic annotations corresponding to perceptually important image regions. The framework can potentially be applied to any multimodal data stream and to any visual domain. To this end, we provide the research community with access to the computational framework.


  • Open Access

    Characterizing the Temporal Dynamics of Information in Visually Guided Predictive Control Using LSTM Recurrent Neural Networks

    by Kamran Binaee, Anna Starynska, Jeff B Pelz, Christopher Kanan, Gabriel Jacob Diaz

    Theories for visually guided action account for online control in the presence of reliable sources of visual information, and predictive control to compensate for visuomotor delay and temporary occlusion. In this study, we characterize the temporal relationship between information integration window and prediction distance using computational models. Subjects were immersed in a simulated environment and attempted to catch virtual balls that were transiently "blanked" during flight. Recurrent neural networks were trained to reproduce subject's gaze and hand movements during blank. The models successfully predict gaze behavior within 3 degrees, and hand movements within 8.5 cm as far as 500 ms in time, with integration window as short as 27 ms. Furthermore, we quantified the contribution of each input source of information to motor output through an ablation study. The model is a proof of concept for prediction as a discrete mapping between information integrated over time and a temporally distant motor output.

  • Auto-simulator Preparation for Research into Assessing the Correlation Between Human Driving Behaviors and Fixation Patterns

    by Mingming Wang, Kate Walders, Martin E. Gordon, Jeff B. Pelz, Susan Farnand

    Autonomous driving has the potential to positively impact the daily life of humans. Techniques such as imaging processing, computer vision, and remote sensing have been highly involved in creating reliable and secure robotic cars. Conversely, the interaction between human perception and autonomous driving has not been deeply explored. Therefore, the analysis of human perception during the cognitive driving task, while making critical driving decisions, may provide great benefits for the study of autonomous driving. To achieve such an analysis, eye movement data of human drivers was collected with a mobile eye-tracker while driving in a automotive simulator built around an actual physical car, that mimics a realistic driving experience. Initial experiments have been performed to investigate the potential correlation between the driving behaviors and fixation patterns of the human driver.


  • Open Access

    SNAG: Spoken Narratives and Gaze Dataset

    by Preethi Vaidyanathan, Emily T. Prud’hommeaux, Jeff B. Pelz, Cecilia O. Alm


  • Eye tracking to evaluate evidence recognition in crime scene investigations

    by Renuka Devi Watalingam, Nicole Richetelli, Jeff B. Pelz, Jacqueline A. Speir


  • Open Access

    Binocular eye tracking calibration during a virtual ball catching task using head mounted display

    by Kamran Binaee, Gabriel Diaz, Jeff Pelz, Flip Phillips

    When tracking the eye movements of an active observer, the quality of the tracking data is continuously affected by physical shifts of the eye-tracker on an observers head. This is especially true for eye-trackers integrated within virtual-reality (VR) helmets. These configurations modify the weight and inertia distribution well beyond that of the eye-tracker alone. Despite the continuous nature of this degradation, it is common practice for calibration procedures to establish eye-to-screen mappings, fixed over the time-course of an experiment. Even with periodic recalibration, data quality can quickly suffer due to head motion. Here, we present a novel post-hoc calibration method that allows for continuous temporal interpolation between discrete calibration events. Analysis focuses on the comparison of fixed vs. continuous calibration schemes and their effects upon the quality of a binocular gaze data to virtual targets, especially with respect to depth. Calibration results were applied to binocular eye tracking data from a VR ball catching task and improved the tracking accuracy especially in the dynamic case.


  • A study of artificial eyes for the measurement of precision in eye-trackers

    by Dong Wang, Fiona B. Mulvey, Jeff B. Pelz, Kenneth Holmqvist

    The precision of an eye-tracker is critical to the correct identification of eye movements and their properties. To measure a system's precision, artificial eyes (AEs) are often used, to exclude eye movements influencing the measurements. A possible issue, however, is that it is virtually impossible to construct AEs with sufficient complexity to fully represent the human eye. To examine the consequences of this limitation, we tested currently used AEs from three manufacturers of eye-trackers and compared them to a more complex model, using 12 commercial eye-trackers. Because precision can be measured in various ways, we compared different metrics in the spatial domain and analyzed the power-spectral densities in the frequency domain. To assess how precision measurements compare in artificial and human eyes, we also measured precision using human recordings on the same eye-trackers. Our results show that the modified eye model presented can cope with all eye-trackers tested and acts as a promising candidate for further development of a set of AEs with varying pupil size and pupil-iris contrast. The spectral analysis of both the AE and human data revealed that human eye data have different frequencies that likely reflect the physiological characteristics of human eye movements. We also report the effects of sample selection methods for precision calculations. This study is part of the EMRA/COGAIN Eye Data Quality Standardization Project.