PID-Guided Partial Alignment for Multimodal Decentralized Federated Learning
Preprint
Abstract
Multimodal decentralized federated learning (DFL) is challenging because agents differ in available modalities and model architectures, yet must collaborate over peer-to-peer (P2P) networks without a central coordinator. Standard multimodal pipelines learn a single shared embedding across all modalities. In DFL, such a monolithic representation induces gradient misalignment between uni- and multimodal agents; as a result, it suppresses heterogeneous sharing and cross-modal interaction. We present PARSE, a multimodal DFL framework that operationalizes partial information decomposition (PID) in a server-free setting. Each agent performs feature fission to factorize its latent representation into redundant, unique, and synergistic slices. P2P knowledge sharing among heterogeneous agents is enabled by slice-level partial alignment: only semantically shareable branches are exchanged among agents that possess the corresponding modality. By removing the need for central coordination and gradient surgery, PARSE resolves uni-/multimodal gradient conflicts, thereby overcoming the multimodal DFL dilemma while remaining compatible with standard DFL constraints. Across benchmarks and agent mixes, PARSE yields consistent gains over task-, modality-, and hybrid-sharing DFL baselines. Ablations on fusion operators and split ratios, together with qualitative visualizations, further demonstrate the efficiency and robustness of the proposed design.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Communication-Efficient Multimodal Federated Learning: Joint Modality and Client SelectionIEEE Transactions on Mobile Computing, 2026
- Decentralized Federated Learning: A Survey and PerspectiveIEEE Internet of Things Journal, 2024
- Cross-Modal Federated Human Activity RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
- Knowledge Distillation and Training Balance for Heterogeneous Decentralized Multi-Modal Learning Over Wireless NetworksIEEE Transactions on Mobile Computing, 2024
- Emotion recognition framework using multiple modalities for an effective human–computer interactionThe Journal of Supercomputing, 2023
- Deep Learning for Intelligent Human–Computer InteractionApplied Sciences, 2022
- Towards Optimal Multi-Modal Federated Learning on Non-IID Data with Hierarchical Gradient Blending2022
- A unified framework for multi-modal federated learningNeurocomputing, 2022
- KU-HAR: An open dataset for heterogeneous human activity recognitionPattern Recognition Letters, 2021
- On the Classification of Emotional Biosignals Evoked While Viewing Affective Pictures: An Integrated Data-Mining-Based Approach for Healthcare ApplicationsIEEE Transactions on Information Technology in Biomedicine, 2010