Fair Concurrent Training of Multiple Models in Federated Learning
- 23 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 33 (6) , 2881-2896
Abstract
Federated learning (FL) enables collaborative learning across multiple clients. In most FL work, all clients train a single learning task. However, the recent proliferation of FL applications may increasingly require multiple FL tasks to be trained simultaneously, sharing clients’ computing resources, which we call Multiple-Model Federated Learning (MMFL). Current MMFL algorithms use naïve average-based client-task allocation schemes that often lead to unfair performance when FL tasks have heterogeneous difficulty levels, as the more difficult tasks may need more client participation to train effectively. Furthermore, in the MMFL setting, we face a further challenge that some clients may prefer training specific tasks to others, and may not even be willing to train other tasks, e.g., due to high computational costs, which may exacerbate unfairness in training outcomes across tasks. We address both challenges by firstly designing FedFairMMFL, a difficulty-aware algorithm that dynamically allocates clients to tasks in each training round, based on the tasks’ current performance levels. We provide guarantees on the resulting task fairness and FedFairMMFL’s convergence rate. We then propose novel auction designs that incentivizes clients to train multiple tasks, so as to fairly distribute clients’ training efforts across the tasks, and extend our convergence guarantees to this setting. We finally evaluate our algorithm with multiple sets of learning tasks on real world datasets, showing that our algorithm improves fairness by improving the final model accuracy and convergence speed of the worst performing tasks, while maintaining the average accuracy across tasks.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- MergeSFL: Split Federated Learning with Feature Merging and Batch Size Regulation2024
- Multi-Model Federated Learning2022
- An Efficient Multi-Model Training Algorithm for Federated Learning2021
- Fairness-Aware Crowdsourcing of IoT Energy Services2021
- An Incentive Mechanism for Cross-Silo Federated Learning: A Public Goods Perspective2021
- FAIR: Quality-Aware Federated Learning with Precise User Incentive and Model Aggregation2021
- Distributed Sensing Using Smart End-User Devices: Pathway to Federated Learning for Autonomous IoT2019
- Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge2019
- An Axiomatic Theory of Fairness in Network Resource Allocation2010
- Generalized α-fair resource allocation in wireless networks2008