Toward Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients
Top Cited Papers
- 13 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Networking
- Vol. 34, 3210-3225
Abstract
In personalized Federated Learning (pFL), high data heterogeneity can cause significant gradient divergence across devices, adversely affecting the learning process. This divergence, especially when gradients from different users form an obtuse angle during aggregation, can negate progress, leading to severe weight and gradient update degradation. To address this issue, we introduce a new approach to pFL design, namely Federated Learning with Layer-wise Aggregation via Gradient Analysis (FedLAG), utilizing the concept of gradient conflict at the layer level. Specifically, when layer-wise gradients of different clients form acute angles, those gradients align in the same direction, enabling updates across different clients toward identifying client-invariant features. Conversely, when layer-wise gradient pairs make create obtuse angles, the layers tend to focus on client-specific tasks. In hindsights, FedLAG assigns layers for personalization based on the extent of layer-wise gradient conflicts. Specifically, layers with gradient conflicts are excluded from the global aggregation process. The theoretical evaluation demonstrates that when integrated into other pFL baselines, FedLAG enhances pFL performance by a certain margin. Therefore, our proposed method achieves superior convergence behavior compared with other baselines. Extensive experiments show that our FedLAG outperforms several state-of-the-art methods and can be easily incorporated with many existing methods to further enhance performance.Keywords
All Related Versions
Funding Information
- Vietnam National Foundation for Science and Technology Development
This publication has 13 references indexed in Scilit:
- Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning2023
- Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaboration2023
- A New Look and Convergence Rate of Federated Multitask Learning With Laplacian RegularizationIEEE Transactions on Neural Networks and Learning Systems, 2022
- A ConvNet for the 2020s2022
- Learn from Others and Be Yourself in Heterogeneous Federated Learning2022
- Model-Contrastive Federated Learning2021
- EMNIST: Extending MNIST to handwritten letters2017
- Deep Residual Learning for Image Recognition2016
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998