Aligning Before Aggregating: Enabling Communication Efficient Cross-Domain Federated Learning via Consistent Feature Extraction
- 19 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 23 (5) , 5880-5896
Abstract
Cross-domain federated learning (FL), where data on local clients come from different domains, is a common case of FL. In such a cross-domain case, features extracted from the raw data of different clients deviate from each other in the feature space, leading to a so-called feature shift. This phenomenon can reduce feature discrimination and degrade the performance of the learned model. However, most existing FL methods are not specifically designed for the cross-domain setting. In this paper, we propose a novel cross-domain FL method named AlignFed. In AlignFed, each client model consists of a personalized feature extractor and a shared lightweight classifier. The feature extractor maps the features to a consistent space by aligning them to identical global target points. Inspired by recent studies in contrastive learning, AlignFed regards points that are uniformly distributed on the hypersphere as global target points. It then pushes features toward global target points of their corresponding classes and away from those of other classes to improve feature discrimination. The shared classifier aggregates knowledge across clients over the consistent feature space, which can mitigate performance degradation caused by feature shift while reducing communication cost. We conduct convergence analysis and perform extensive experiments to evaluate AlignFed.Keywords
Funding Information
- National Natural Science Foundation of China (61976012)
This publication has 35 references indexed in Scilit:
- Neural Map Prior for Autonomous Driving2023
- SphereFed: Hyperspherical Federated Learning2022
- Targeted Supervised Contrastive Learning for Long-Tailed Recognition2022
- Embracing the disharmony in medical imaging: A Simple and effective framework for domain adaptationMedical Image Analysis, 2021
- Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification2021
- The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesionsScientific Data, 2018
- Adversarial Discriminative Domain Adaptation2017
- Deep Residual Learning for Image Recognition2016
- Epidemiology and Economic Burden of Nonmelanoma Skin CancerFacial Plastic Surgery Clinics of North America, 2012
- Nonmelanoma Skin CancerFacial Plastic Surgery Clinics of North America, 2009