TinyReptile: TinyML with Federated Meta-Learning
Authors:
- 18 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Tiny machine learning (TinyML) is a rapidly growing field aiming to democratize machine learning (ML) for resource-constrained microcontrollers (MCUs). Given the pervasiveness of these tiny devices, it is inherent to ask whether TinyML applications can benefit from aggregating their knowledge. Federated learning (FL) enables decentralized agents to jointly learn a global model without sharing sensitive local data. However, a common global model may not work for all devices due to the complexity of the actual deployment environment and the heterogeneity of the data available on each device. In addition, the deployment of TinyML hardware has significant computational and communication constraints, which traditional ML fails to address. Considering these challenges, we propose TinyReptile, a simple but efficient algorithm inspired by meta-learning and online learning, to collaboratively learn a solid initialization for a neural network (NN) across tiny devices that can be quickly adapted to a new device with respect to its data. We demonstrate TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM. The evaluations on various TinyML use cases confirm a resource reduction and training time saving by at least two factors compared with baseline algorithms with comparable performance.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- A review on TinyML: State-of-the-art and prospectsJournal of King Saud University Computer and Information Sciences, 2022
- TinyML Meets IoT: A Comprehensive SurveyInternet of Things, 2021
- A TinyML Platform for On-Device Continual Learning With Quantized Latent ReplaysIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2021
- On-Device Training of Machine Learning Models on Microcontrollers With a Look at Federated Learning2021
- TinyOL: TinyML with Online-Learning on Microcontrollers2021
- A Collaborative Learning Framework via Federated Meta-Learning2020
- End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things2020
- Machine learning for streaming dataACM SIGKDD Explorations Newsletter, 2019
- Robust and Communication-Efficient Federated Learning From Non-i.i.d. DataIEEE Transactions on Neural Networks and Learning Systems, 2019
- The Omniglot challenge: a 3-year progress reportCurrent Opinion in Behavioral Sciences, 2019