Device-Performance-Driven Heterogeneous Multiparty Learning for Arbitrary Images
- 13 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (10) , 14932-14944
Abstract
Multiparty learning (MPL) is an emerging framework for privacy-preserving collaborative learning. It enables individual devices to build a knowledge-shared model and remaining sensitive data locally. However, with the continuous increase of users, the heterogeneity gap between data and equipment becomes wider, which leads to the problem of model heterogeneous. In this article, we concentrate on two practical issues: data heterogeneous problem and model heterogeneous problem, and propose a novel personal MPL method named device-performance-driven heterogeneous MPL (HMPL). First, facing the data heterogeneous problem, we focus on the problem of various devices holding arbitrary data sizes. We introduce a heterogeneous feature-map integration method to adaptively unify the various feature maps. Meanwhile, to handle the model heterogeneous problem, as it is essential to customize models for adapting to the various computing performances, we propose a layer-wise model generation and aggregation strategy. The method can generate customized models based on the device’s performance. In the aggregation process, the shared model parameters are updated through the rules that the network layers with the same semantics are aggregated with each other. Extensive experiments are conducted on four popular datasets, and the result demonstrates that our proposed framework outperforms the state of the art (SOTA).Keywords
Funding Information
- Natural Science Foundation of China (62036006, 62276200)
- Key-Area Research and Development Program of Guangdong Province (2020B090921001)
This publication has 28 references indexed in Scilit:
- Federated Machine LearningACM Transactions on Intelligent Systems and Technology, 2019
- Path Aggregation Network for Instance Segmentation2018
- Feature Pyramid Networks for Object Detection2017
- Pyramid Scene Parsing Network2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Challenges and Opportunities in Edge Computing2016
- Edge Computing: Vision and ChallengesIEEE Internet of Things Journal, 2016
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Large-Scale Machine Learning with Stochastic Gradient Descent2010
- On the Relationship between Classical Grid Search and Probabilistic RoadmapsThe International Journal of Robotics Research, 2004