Unsupervised Gaze Representation Learning by Switching Features
- 19 August 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (1) , 62-78
Abstract
It is prevalent to leverage unlabeled data to train deep learning models when it is difficult to collect large-scale annotated datasets. However, for 3D gaze estimation, most existing unsupervised learning methods face challenges in distinguishing subtle gaze-relevant information from dominant gaze-irrelevant information. To address this issue, we propose an unsupervised learning framework to disentangle the gaze-relevant and the gaze-irrelevant information, by seeking the shared information of a pair of input images with the same gaze and with the same eye respectively. Specifically, given two images, the framework finds their shared information by first encoding the images into two latent features via two encoders and then switching part of the features before feeding them to the decoders for image reconstruction. We theoretically prove that the proposed framework is able to encode different information into different parts of the latent feature if we properly select the training image pairs and their shared information. Based on the framework, we derive Cross-Encoder and Cross-Encoder++ to learn gaze representation from the eye images and face images, respectively. Experiments on public gaze datasets demonstrate that the Cross-Encoder and Cross-Encoder++ outperform the competitive methods. The ablation study quantitatively and qualitatively shows that the gaze feature is successfully extracted.Keywords
Funding Information
- National Natural Science Foundation of China (U2336213, 62176248)
This publication has 61 references indexed in Scilit:
- ETH-XGaze: A Large Scale Dataset for Gaze Estimation Under Extreme Head Pose and Gaze Variation2020
- Towards End-to-End Video-Based Eye-Tracking2020
- Deep Multitask Gaze Estimation with a Constrained Landmark-Gaze Model2019
- Appearance-Based Gaze Estimation via Evaluation-Guided Asymmetric Regression2018
- Deep Clustering for Unsupervised Learning of Visual Features2018
- Disentangling Factors of Variation with Cycle-Consistent Variational Auto-encoders2018
- Deep Pictorial Gaze Estimation2018
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Eye Tracking and Eye-Based Human–Computer Interaction2014
- Elements of Information Theory2001