Client-Unbiased Skeletal Action Recognizer in Federated Learning
- 14 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 4908-4922
Abstract
Edge sensor devices generate vast amounts of user data, but centralized processing poses privacy risks. Federated Learning addresses this by decentralizing training. However, applying Federated Learning directly to skeleton videos fails to preserve motion dynamics and suffers from client heterogeneity bias. To address these limitations, we propose CSAR—a Client-Unbiased Skeletal Action Recognizer for Federated Learning— which tackles two core challenges: motion dynamics preservation and classifier bias mitigation. Specifically, CSAR employs a Model Calibration Loss during client training to align client-server representations and reduce drift. On the server, it generates class-balanced spatiotemporal federated features through Prototypical Gaussian Sampling, subsequently refined via a Motion-aware Differential Loss to capture kinematic properties. These features enable retraining of a globally debiased recognizer that achieves accuracy comparable to real-data-trained models. Further stabilization is achieved through Knowledge Matching, which enhances global understanding. Experiments under natural and label heterogeneity confirm that CSAR outperforms state-of-the-art methods.Keywords
Funding Information
- National Natural Science Foundation of China (62222207, 62427808, 62332010)
This publication has 44 references indexed in Scilit:
- Federated Self-supervised Learning for Video Understanding2022
- Human Action Recognition and Prediction: A SurveyInternational Journal of Computer Vision, 2022
- A federated learning system with enhanced feature extraction for human activity recognitionKnowledge-Based Systems, 2021
- Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack2021
- Model-Contrastive Federated Learning2021
- Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition2020
- A model-based gait recognition method with body pose and human prior knowledgePattern Recognition, 2020
- Actional-Structural Graph Convolutional Networks for Skeleton-Based Action Recognition2019
- Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks2017
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis2016