Gaze Estimation Based on Attention Mechanism Combined With Temporal Network
Open Access
- 18 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 107150-107159
Abstract
Due to the continuous and dynamic nature of gaze estimation, the true gaze point at each moment is closely related to the previous moment. Simply detecting individual frames of facial images cannot yield accurate gaze information. In current CNN-based gaze estimation methods, the effective utilization of eye movement temporal information and the ability to capture global relationships in the feature extraction process remain problematic. Addressing these concerns, this paper proposes a novel gaze estimation framework, named FE-net, which incorporates a temporal network. This framework introduces channel attention modules and self-attention modules, enhancing the comprehensive utilization of extracted features and reinforcing the contribution of valuable regions to gaze estimation. We further integrate an RNN structure to learn the temporal dynamics of eye movement processes, significantly improving gaze direction prediction accuracy. This framework predicts the gaze directions of left and right eyes separately using monocular and facial features and computes the overall gaze direction. FE-net achieves state-of-the-art accuracy of 3.19° and 3.16° on the EVE dataset and the MPIIFaceGaze dataset, respectively.Keywords
Funding Information
- National Natural Science Foundation of China (62001035)
- Research and Development Program of Beijing Municipal Education Commission (KM202111232018)
- Natural Science Foundation of Beijing (4232044)
This publication has 34 references indexed in Scilit:
- Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression2018
- MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze EstimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- It’s Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation2017
- Deep Residual Learning for Image Recognition2016
- Eye Tracking for Everyone2016
- Appearance-based gaze estimation in the wild2015
- Geometric Generative Gaze Estimation (G3E) for Remote RGB-D Cameras2014
- Point of Gaze Estimation through Corneal Surface Reflection in an Active Illumination Environment2012
- General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal ReflectionsIEEE Transactions on Biomedical Engineering, 2006
- Long Short-Term MemoryNeural Computation, 1997