HPFL: Federated Learning by Fusing Multiple Sensor Modalities with Heterogeneous Privacy Sensitivity Levels
Authors:
Open Access
- 10 October 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 5-14
Abstract
Solving classification problems to understand multi-modality sensor data has become popular, but rich-media sensors, e.g., RGB cameras and microphones, are privacy-invasive. Though existing Federated Learning (FL) algorithms allow clients to keep their sensor data private, they suffer from degraded performance, particularly lower classification accuracy and longer training time, than centralized learning. We propose a Heterogeneous Privacy Federated Learning (HPFL) paradigm to capitalize on the information in the privacy insensitive data (such as mmWave point clouds) while keeping the privacy sensitive data (such as RGB images) private because sensor data are of diverse sensitivity levels. We evaluate the HPFL paradigm on two representative classification problems: semantic segmentation and emotion recognition. Extensive experiments demonstrate that the HPFL paradigm outperforms: (i) the popular FedAvg by 18.20% in foreground accuracy (semantic segmentation) and 4.20% in F1-score (emotion recognition) under non-i.i.d. sample distributions and (ii) the state-of-the-art FL algorithms by 12.40%--17.70% in foreground accuracy and 2.54%--4.10% in F1-score.Keywords
This publication has 38 references indexed in Scilit:
- Robust and Communication-Efficient Federated Learning From Non-i.i.d. DataIEEE Transactions on Neural Networks and Learning Systems, 2019
- CMFL: Mitigating Communication Overhead for Federated Learning2019
- ICNet for Real-Time Semantic Segmentation on High-Resolution Images2018
- A Survey on Deep Transfer Learning2018
- A survey on deep learning techniques for image and video semantic segmentationApplied Soft Computing, 2018
- Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and ChallengesIEEE Signal Processing Magazine, 2018
- MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes2017
- Fully Convolutional Instance-Aware Semantic Segmentation2017
- Learning Deconvolution Network for Semantic Segmentation2015
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008