‘Disengage AND Integrate’: Personalized Causal Network for Gaze Estimation
- 6 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 3733-3747
Abstract
Gaze estimation task aims to predict a 3D gaze direction or a 2D gaze point given a face or eye image. To improve generalization of gaze estimation models to unseen new users, existing methods either disentangle personalized information of all subjects from their gaze features, or integrate unrefined personalized information into blended embeddings. Their methodologies are not rigorous whose performance is still unsatisfactory. In this paper, we put forward a comprehensive perspective named ‘Disengage AND Integrate’ to deal with personalized information, which elaborates that for specified users, their irrelevant personalized information should be discarded while relevant one should be considered. Accordingly, a novel Personalized Causal Network (PCNet) for generalizable gaze estimation has been proposed. The PCNet adopts a two-branch framework, which consists of a subject-deconfounded appearance sub-network (SdeANet) and a prototypical personalization sub-network (ProPNet). The SdeANet aims to explore causalities among facial images, gazes, and personalized information and extract a subject-invariant appearance-aware feature of each image by means of causal intervention. The ProPNet aims to characterize customized personalization-aware features of arbitrary users with the help of a prototype-based subject identification task. Furthermore, our whole PCNet is optimized in a hybrid episodic training paradigm, which further improve its adaptability to new users. Experiments on three challenging datasets over within-domain and cross-domain gaze estimation tasks demonstrate the effectiveness of our method.Keywords
Funding Information
- Chinese Aeronautical Establishment (2023Z0710M5001)
- National Natural Science Foundation of China (62376021, 62271042)
This publication has 50 references indexed in Scilit:
- Causal Intervention for Brain Tumor Segmentation2024
- Episode-based personalization network for gaze estimation without calibrationNeurocomputing, 2022
- Domain Adaptation Gaze Estimation by Embedding with Prediction Consistency2021
- ETH-XGaze: A Large Scale Dataset for Gaze Estimation Under Extreme Head Pose and Gaze Variation2020
- Appearance-Based Gaze Estimation Using Dilated-Convolutions2019
- RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments2018
- A Hierarchical Generative Model for Eye Image Synthesis and Eye Gaze Estimation2018
- Adversarial Discriminative Domain Adaptation2017
- Unconstrained Gaze Estimation Using Random Forest Regression Voting2017
- Eye Tracking for Everyone2016