Federated Class-Incremental Learning
Authors:
Top Cited Papers
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 10154-10163
Abstract
Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global class-incremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%~15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github.com/conditionWang/FCIL.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation2020
- Mnemonics Training: Multi-Class Incremental Learning Without Forgetting2020
- Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted AggregationIEEE Transactions on Neural Networks and Learning Systems, 2019
- Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular CommunicationsIEEE Transactions on Communications, 2019
- Two-Stage Label Embedding via Neural Factorization Machine for Multi-Label ClassificationProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Large Scale Incremental Learning2019
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- Learning Without Forgetting2016
- Deep Residual Learning for Image Recognition2016
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998