CBFL: A Communication-Efficient Federated Learning Framework From Data Redundancy Perspective

Authors:
Publisher website
Google Scholar
Abstract
Federated learning (FL) is an emerging machinelearning framework, which enables multiple mobile users to collaboratively train a global model without uploading their local sensitive data. Due to limited network bandwidth, communication efficiency has become a significant bottleneck for the implementation of FL. Existing works attempt to improve this situation by reducing the total bits transferred for each client update via data compression. However, these research works are only from the perspective of update parameters to reduce the transmission of redundant parameters. They do not explore the intrinsic reasons for redundant parameters. In this article, we propose a coreset-based FL (CBFL) framework. Instead of training model on full datasets with a regular network model, CBFL uses a much smaller well-matched evolutionary network model on coreset. CBFL indirectly reduces the total transmission bits for each client while achieving a similar accuracy as training with full datasets. CBFL includes a novel distributed coreset construction and adaptive model evolution algorithms. The network model is adaptively adjusted during this process, which dynamically removes the least important connections from the current model. Experimental results with various datasets and models show that CBFL is able to find an optimized evolutionary model that has about 10% of the total number of connections in the original regular model while only about 2% degradation in model accuracy.
Funding Information
  • National Natural Science Foundation of China (61772094)
  • Entrepreneurship and Innovation Support Plan for overseas Students in Chongqing (CX2019094)
  • Central University Fund (2020CDJQY-A019)

This publication has 15 references indexed in Scilit: