Robust Federated Fine-Tuning in Heterogeneous Networks with Unreliable Connections: An Aggregation View
Preprint
Abstract
Federated Fine-Tuning (FFT) has attracted growing interest as it leverages both server- and client-side data to enhance global model generalization while preserving privacy, and significantly reduces the computational burden on edge devices by avoiding training from scratch. Despite these advantages, FFT performance is often degraded by unreliable server-client connections and heterogeneous client data distributions. Most existing methods assume homogeneous network conditions or require prior knowledge of connection failures. However, these assumptions are impractical in real-world networks characterized by diverse communication standards (e.g., wired, Wi-Fi, 4G, and 5G) and heterogeneous failure patterns. To address these limitations, we propose FedAuto, a novel FFT framework that mitigates the combined effects of connection failures and data heterogeneity via adaptive aggregation. FedAuto operates without prior knowledge of network conditions or modifications to existing infrastructure, enabling seamless plug-and-play deployment. Moreover, we establish a rigorous convergence guarantee for FedAuto. By adopting a novel per-round aggregation perspective, our analysis removes the need for assumptions on connection failures probabilities or client selection strategies commonly imposed in prior work, and guarantees convergence of FedAuto for each individual realization, providing a stronger theoretical assurance. Extensive experiments demonstrate that FedAuto consistently outperforms state-of-the-art baselines under diverse connection failure scenarios for both full-parameter and partial-parameter fine-tuning (e.g., LoRA), and even surpasses strategies that rely on complex communication resource optimization.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- A Federated Fine-Tuning Paradigm of Foundation Models in Heterogenous Wireless Networks2025
- Towards Building The Federatedgpt: Federated Instruction Tuning2024
- Towards Bias Correction of FedAvg over Nonuniform and Time-Varying Communications2023
- Federated Learning Resource Optimization and Client Selection for Total Energy Minimization Under Outage, Latency, and Bandwidth Constraints With Partial or No CSIIEEE Open Journal of the Communications Society, 2023
- Federated Learning with GAN-Based Data Synthesis for Non-IID Clients2023
- Federated Learning Under Intermittent Client Availability and Time-Varying Communication ConstraintsIEEE Journal of Selected Topics in Signal Processing, 2022
- Quantized Federated Learning Under Transmission Delay and Outage ConstraintsIEEE Journal on Selected Areas in Communications, 2021
- Federated Learning in Unreliable and Resource-Constrained Cellular Wireless NetworksIEEE Transactions on Communications, 2021
- Federated Learning for Internet of Things: A Comprehensive SurveyIEEE Communications Surveys & Tutorials, 2021
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998