TinyMetaFed: Efficient Federated Meta-Learning for TinyML
Preprint
Abstract
The field of Tiny Machine Learning (TinyML) has made substantial advancements in democratizing machine learning on low-footprint devices, such as microcontrollers. The prevalence of these miniature devices raises the question of whether aggregating their knowledge can benefit TinyML applications. Federated meta-learning is a promising answer to this question, as it addresses the scarcity of labeled data and heterogeneous data distribution across devices in the real world. However, deploying TinyML hardware faces unique resource constraints, making existing methods impractical due to energy, privacy, and communication limitations. We introduce TinyMetaFed, a model-agnostic meta-learning framework suitable for TinyML. TinyMetaFed facilitates collaborative training of a neural network initialization that can be quickly fine-tuned on new devices. It offers communication savings and privacy protection through partial local reconstruction and Top-P% selective communication, computational efficiency via online learning, and robustness to client heterogeneity through few-shot learning. The evaluations on three TinyML use cases demonstrate that TinyMetaFed can significantly reduce energy consumption and communication overhead, accelerate convergence, and stabilize the training process.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Meta-federated learning2024
- TinyReptile: TinyML with Federated Meta-Learning2023
- Incremental Online Learning Algorithms Comparison for Gesture and Visual Smart Sensors2022
- On-Device Training of Machine Learning Models on Microcontrollers with Federated LearningElectronics, 2022
- Privacy-Preserving Federated Learning for Industrial Edge Computing via Hybrid Differential Privacy and Adaptive CompressionIEEE Transactions on Industrial Informatics, 2021
- Dynamic Sampling and Selective Masking for Communication-Efficient Federated LearningIEEE Intelligent Systems, 2021
- A Survey on Federated Learning for Resource-Constrained IoT DevicesIEEE Internet of Things Journal, 2021
- End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020