MAIR: Model Agnostic Instance Reweighing for Heterogeneous Federated Learning
- 22 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (3) , 4241-4252
Abstract
Federated learning (FL) enables collaborative training on decentralized data while preserving the data owners’ privacy, under the orchestration of a central server. FL has seen tremendous growth and advancements in recent years. Despite its progress, FL faces a significant challenge raised by data heterogeneity, leading to a slower convergence rate and a larger performance gap compared to centralized training. In this work, we empirically reveal that direct applying empirical risk minimizing (ERM) on skewed client training data causes the client model suffers from biased predictions towards majority classes. To address this problem, we propose a model agnostic instance reweighing method (MAIR). At a coarse-grained level, MAIR adjusts the logits predictions for each class to counteract the data heterogeneity. At a fine-grained level, it dynamically reweighs the importance of individual training samples with a predictive meta network. As a results, MAIR prevents client models from over-fitting on heterogeneous data and therefore substantially reduces client drift. Theoretically, we justify its non-convex convergence property. Extensive experiments demonstrate that MAIR reliably speeds up convergence and improves the quality of global models, outperforming its best competitor by a clear margin. It notably delivers $8.3\%$ improvements on ImageNet subset and achieves $67.6\%$ energy footprint reduction on CIFAR-100 over the FedAvg baseline. Our findings also suggest that improving the performance of FL-trained models necessitates rethinking clients’ local optimization objectives, and ERM should thus no longer be viewed as a de facto standard in FL under data heterogeneity.
Keywords
Funding Information
- Technology Development Fund of Macao S.A.R (0123/2022/AFJ)
- Technology Development Fund of Macao S.A.R (0081/2022/A2)
- National Natural Science Foundation of China (62376263)
- Basic and Applied Basic Research Foundation of Guangdong Province (2023B1515130002)
- Natural Science Foundation of Guangdong Province (2024A1515030209)
- Science, Technology and Innovation Commission of Shenzhen Municipality (JCYJ20230807140507015)
- SICC
- SKL-IOTSC
- Universidade de Macau
This publication has 21 references indexed in Scilit:
- Advanced Zero-Shot Learning (AZSL) Framework for Secure Model Generalization in Federated LearningIEEE Access, 2024
- Model-Contrastive Federated Learning2021
- Class-Balanced Loss Based on Effective Number of Samples2019
- The EU General Data Protection Regulation (GDPR)2017
- Learning Deep Representation for Imbalanced Classification2016
- Microsoft COCO: Common Objects in Context2014
- A survey on feature selection methodsComputers and Electrical Engineering, 2014
- Bayesian Reasoning and Machine Learning2012
- ImageNet: A large-scale hierarchical image database2009
- The Nature of Statistical Learning Theory1995