Label-Specific Feature Based Multi-Label Neural Network for Federated Learning
Authors:
- 28 February 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 130-136
Abstract
Federated learning (FL) enables the collaborative training of deep learning models across decentralized data sources (i.e., clients) while preserving data privacy by not requiring direct access to clients' training data. However, in the case of multi-label learning, the label correlations across clients are often diverse, which can disrupt the classification process. Direct application of an end-to-end classification model on each client may lead to inaccurate multi-label predictions due to these varying label dependencies across different clients. To resolve this issue, this paper begins by projecting the data and label information into a global feature space. Next, for each class, it retrieves the specific features related to that label from the global feature space, proactively mitigating inconsistent label correlations by focusing on label-specific features. Specifically, this paper proposes a label-specific approach: Label-Specific Feature-Based Multi-Label Federated Transformer (LSFT), where a distinct classification model is developed for each class on the clients. These models are constructed based on the Transformer and design label-specific classifier for each class based on the extracted features from the Transformer, which extract unique, label-specific discriminative features to alleviate the dependency on label correlation. Once gathered, these parameters are aggregated at the global model for each class, ensuring that the label correlations are properly aligned and minimizing interference, facilitating more effective multi-label federated learning framework. Through extensive experiments on various multi-label datasets (e.g., PASCAL VOC, MS-COCO), we demonstrate that the proposed model achieves competitive performance and outperforms standard federated learning methods in multi-label federated learning settings.Keywords
This publication has 15 references indexed in Scilit:
- Label-Specific Time–Frequency Energy-Based Neural Network for Instrument RecognitionIEEE Transactions on Cybernetics, 2024
- Multi-dimensional multi-label classification: Towards encompassing heterogeneous label spaces and multi-label annotationsPattern Recognition, 2023
- FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction2022
- Model-Contrastive Federated Learning2021
- General Multi-label Image Classification with Transformers2021
- Deep-LIFT: Deep Label-Specific Feature Learning for Image AnnotationIEEE Transactions on Cybernetics, 2021
- A review of applications in federated learningComputers & Industrial Engineering, 2020
- Deep Residual Learning for Image Recognition2016
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Microsoft COCO: Common Objects in Context2014