Privacy-Preserving On-Screen Activity Recognition via One-Shot Federated Learning
Open Access
- 28 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 201625-201644
Abstract
Preserving user privacy while monitoring on-screen activity is a growing challenge in remote and decentralized work environments. In this paper, we propose a novel one-shot federated learning (FL) framework that enables secure, low-overhead, and privacy-preserving on-screen activity recognition. Conventional FL methods require multiple communication rounds between clients and a central server. In contrast, our one-shot FL approach performs model aggregation in a single round, significantly reducing communication costs and latency, making it ideal for bandwidth-constrained and resource-limited environments. Using a custom dataset of 2,300 screenshots from common applications (e.g., Coursera, YouTube, Amazon), we train Deep Convolutional Neural Network (DCNN) models such as VGG16, VGG19, and InceptionResNetV2 locally on client devices. Sensitive visual data is never shared, and the screenshots are processed and discarded post-inference, ensuring strict data confidentiality. We further enhance privacy by integrating differential privacy mechanisms. To further improve learning effectiveness under different data distributions, we evaluate two aggregation techniques, FedAvg for IID and FedProx for non-IID settings. Experimental evaluations show that our approach achieves up to 99.1% classification accuracy (MobileNet) while reducing communication overhead by over 80% compared to traditional FL. This work highlights the effectiveness and scalability of one-shot FL for secure, real-time activity tracking in privacy-sensitive applications.Keywords
This publication has 35 references indexed in Scilit:
- Privacy-sensitive federated learning for cross-domain adaptation: The Mamba-MoE approachResults in Engineering, 2025
- IBPL: Information Bottleneck-based Prompt Learning for graph out-of-distribution detectionNeural Networks, 2025
- Perturbation defense ultra high-speed weak target recognitionEngineering Applications of Artificial Intelligence, 2024
- Image Classification Using Federated Learning2024
- Attention monitoring of students during online classes using XGBoost classifierComputers and Education: Artificial Intelligence, 2023
- Privacy-Preserving On-Screen Activity Tracking and Classification in E-Learning Using Federated LearningIEEE Access, 2023
- TASTA: Text‐Assisted Spatial and Temporal Attention Network for Video Question AnsweringAdvanced Intelligent Systems, 2023
- Towards efficient communications in federated learning: A contemporary surveyJournal of the Franklin Institute, 2023
- ResLNet: deep residual LSTM network with longer input for action recognitionFrontiers of Computer Science, 2022
- Fusion Learning: A One Shot Federated Learning2020