TransGOP-R: Transformer-Based Real-World Gaze Object Prediction
- 3 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1689-1703
Abstract
The goal of gaze object prediction (GOP) is to predict human gaze objects and categories. However, existing methods require additional head priors or filter the results before evaluation, which is an obstacle for real-world applications. To this end, this paper proposes a Transformer-based Gaze Object Prediction under Real-world setting (TransGOP-R), which does not rely on any head prior input and evaluates end-to-end. We first design a head location module to generate human head location information from a head query. Then, an error analysis demonstrates that the primary error source of the existing GOP model is in gaze estimation, which is caused by the difficulty in predicting gaze points by directly regressing heatmaps. Therefore, we introduce cone prediction into the model training stage, allowing the middle-layer features of the gaze regressor to build the relationship between the target human and objects before regressing the gaze point. An oriented gradient mechanism is proposed in this process to ensure the object detection performance is not affected by cone information. Finally, we conducted very detailed and sufficient experiments to verify the superiority of our method on the GOO-Synth and GOO-Real datasets. At the same time, we also achieve advantages compared to the human-target gaze estimation methods on the GazeFollowing, VideoAttentionTarget, and ChildPlay datasets.Keywords
Funding Information
- Key Research and Development Program of Shaanxi (2024GX-YBXM-051)
- National Natural Science Foundation of China (62401447)
- National Natural Science Foundation of China (62306101)
This publication has 65 references indexed in Scilit:
- Exploring Plain Vision Transformer Backbones for Object Detection2022
- Depth-aware gaze-following via auxiliary networks for roboticsEngineering Applications of Artificial Intelligence, 2022
- End-to-End Object Detection with Transformers2020
- Believe It or Not, We Know What You Are Looking At!2019
- Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression2018
- Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliency2018
- Deep Pictorial Gaze Estimation2018
- Microsoft COCO: Common Objects in Context2014
- Looking and ActingVision and eye movements in natural behaviour2009
- The eyes have it: the neuroethology, function and evolution of social gazeNeuroscience & Biobehavioral Reviews, 2000