Rethinking Self-balancing Federated Learning under Non-uniformParameter Influence Density

Authors:
  • GFGuangsheng Feng
    Guangsheng Feng
  • KCKaixuan Cong
    Kaixuan Cong
  • HLHongwu Lv
    Hongwu Lv
  • WZWenqi Zheng
    Wenqi Zheng
  • KZKaiqi Zhang
    Kaiqi Zhang
  • ZWZihui Wang
    Zihui Wang

Abstract
Federated Learning (FL), as a distributed machine learning paradigm, has been persistentlychallenged by data heterogeneity. Most existing studies attempt to mitigate client drift byenforcing model similarity in parameter space. However, non-uniform parameter influence onthe loss function induces an ease-seeking behavior in model optimization, resulting in aninconsistency between model similarity and actual performance. To address this issue, wepropose FedPID, a PID-aware self-balancing framework based on Parameter Influence Density (PID), to optimize model performance under data heterogeneity. In local training, we propose an Adaptive Compensation Loss (ACL), which combines count-based class compensation with batch-level PID modulation to prevent overly aggressive updates along high-influence parameter directions. In global aggregation, we introduce a Calibrated Aggregation Mechanism (CAM) to recalibrate aggregation weights according to PID-weighted update fluctuation and reduce the contribution of unstable clients. Extensive experiments demonstrate that FedPID achieves an average improvement of 2.29 percentage points over state-of-the-art methods across five commonly used visual and text datasets under varying levels of statistical heterogeneity