Personalized Federated Learning for Egocentric Video Gaze Estimation With Comprehensive Parameter Freezing

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Egocentric video gaze estimation ipredicts where an individual looks in first-person videos providing an essential understanding of user attention. However, it must accommodate the variability in individuals’ gaze patterns and support learning in scenarios in which data are distributed across multiple devices. To overcome these challenges, we introduce a Personalized Federated Learning (PFL) framework with Comprehensive Parameters Freezing (FedCPF) that improves gaze estimation accuracy and personalization in distributed settings. Our approach employs a transformer-based backbone to capture temporal dependencies and spatial correlations in egocentric video frames.Within a PFL setting, only those parameters that exhibit the most significantly updated parameters during local training are selected and subsequently frozen, developing tailored client models that preserve high-impact features. By freezing these key parameters, FedCPF reduces the communication overhead and mitigates issues from heterogeneous data distributions while allowing clients to fine-tune models for their unique gaze behaviors. We validate FedCPF through extensive experiments on the EGTEA Gaze+ and Ego4D datasets. The experimental results demonstrate that our comprehensive parameter freezing strategy effectively balances global collaboration and local adaptation, making FedCPF a promising solution for accurate, adaptive gaze estimation in federated learning.
Funding Information
  • Japan Society for the Promotion of Science (JSPS) Kakenhi (JP23K21676)
  • Japan Society for the Promotion of Science (JSPS) Kakenhi (JP23K11211)
  • Japan Science and Technology Agency (JST) Broadening Opportunities for Outstanding young researchers and doctoral students in STrategic areas (BOOST), Japan (JPMJBS2426)

This publication has 22 references indexed in Scilit: