Dynamic Topology Optimization for Non-IID Data in Decentralized Learning
Preprint
Abstract
Decentralized learning (DL) enables a set of nodes to train a model collaboratively without central coordination, offering benefits for privacy and scalability. However, DL struggles to train a high accuracy model when the data distribution is non-independent and identically distributed (non-IID) and when the communication topology is static. To address these issues, we propose Morph, a topology optimization algorithm for DL. In Morph, nodes adaptively choose peers for model exchange based on maximum model dissimilarity. Morph maintains a fixed in-degree while dynamically reshaping the communication graph through gossip-based peer discovery and diversity-driven neighbor selection, thereby improving robustness to data heterogeneity. Experiments on CIFAR-10 and FEMNIST with up to 100 nodes show that Morph consistently outperforms static and epidemic baselines, while closely tracking the fully connected upper bound. On CIFAR-10, Morph achieves a relative improvement of 1.12x in test accuracy compared to the state-of-the-art baselines. On FEMNIST, Morph achieves an accuracy that is 1.08x higher than Epidemic Learning. Similar trends hold for 50 node deployments, where Morph narrows the gap to the fully connected upper bound within 0.5 percentage points on CIFAR-10. These results demonstrate that Morph achieves higher final accuracy, faster convergence, and more stable learning as quantified by lower inter-node variance, while requiring fewer communication rounds than baselines and no global knowledge.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- CCBNet : Confidential Collaborative Bayesian Networks Inference2026
- Share Secrets for Privacy: Confidential Forecasting with Vertical Federated Learning2025
- DFL topology optimization based on peer weighting mechanism and graph neural network in digital twin platformComplex & Intelligent Systems, 2025
- Impact of network topology on the performance of Decentralized Federated LearningComputer Networks, 2024
- A state-of-the-art survey on solving non-IID data in Federated LearningFuture Generation Computer Systems, 2022
- A Triangle Inequality for Cosine Similarity2021
- Decentralized Federated Learning Preserves Model and Data Privacy2021
- Gossip Learning as a Decentralized Alternative to Federated Learning2019
- Gossip learning with linear models on fully distributed dataConcurrency and Computation: Practice and Experience, 2012
- Brahms: Byzantine resilient random membership samplingComputer Networks, 2009