Abstract
Preserving user privacy while monitoring on-screen activity is a growing challenge in remote and decentralized work environments. In this paper, we propose a novel one-shot federated learning (FL) framework that enables secure, low-overhead, and privacy-preserving on-screen activity recognition. Conventional FL methods require multiple communication rounds between clients and a central server. In contrast, our one-shot FL approach performs model aggregation in a single round, significantly reducing communication costs and latency, making it ideal for bandwidth-constrained and resource-limited environments. Using a custom dataset of 2,300 screenshots from common applications (e.g., Coursera, YouTube, Amazon), we train Deep Convolutional Neural Network (DCNN) models such as VGG16, VGG19, and InceptionResNetV2 locally on client devices. Sensitive visual data is never shared, and the screenshots are processed and discarded post-inference, ensuring strict data confidentiality. We further enhance privacy by integrating differential privacy mechanisms. To further improve learning effectiveness under different data distributions, we evaluate two aggregation techniques, FedAvg for IID and FedProx for non-IID settings. Experimental evaluations show that our approach achieves up to 99.1% classification accuracy (MobileNet) while reducing communication overhead by over 80% compared to traditional FL. This work highlights the effectiveness and scalability of one-shot FL for secure, real-time activity tracking in privacy-sensitive applications.

This publication has 35 references indexed in Scilit: