Haoyu Ren
Research Subjects
Haoyu Ren has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Darko Anicic
Siemens AG
6 shared publications - Thomas A. Runkler
Siemens AG, Munich, Germany
6 shared publications - Xue Li
The University of Queensland
3 shared publications Darko Anicic
Siemens AG, Germany
1 shared publication- Kirill Dorofeev
Siemens AG
1 shared publication - Roland Eckl
Siemens AG
1 shared publication
Latest Publications
- 1 January 2025
- Published by Springer Nature
- p. 126-137
TinyMetaFed: Efficient Federated Meta-learning for TinyML
- 1 January 2025
- Published by Springer Nature
- p. 126-137
TinyMetaFed: Efficient Federated Meta-learning for TinyML
- Open Access
- 10 June 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 23 (4) , 1-32
On-device Online Learning and Semantic Management of TinyML Systems
Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction.
- 10 June 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 23 (4) , 1-32
Open AccessOn-device Online Learning and Semantic Management of TinyML Systems
Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction.
- Open Access
TinyMetaFed: Efficient Federated Meta-Learning for TinyML
The field of Tiny Machine Learning (TinyML) has made substantial advancements in democratizing machine learning on low-footprint devices, such as microcontrollers. The prevalence of these miniature devices raises the question of whether aggregating their knowledge can benefit TinyML applications. Federated meta-learning is a promising answer to this question, as it addresses the scarcity of labeled data and heterogeneous data distribution across devices in the real world. However, deploying TinyML hardware faces unique resource constraints, making existing methods impractical due to energy, privacy, and communication limitations. We introduce TinyMetaFed, a model-agnostic meta-learning framework suitable for TinyML. TinyMetaFed facilitates collaborative training of a neural network initialization that can be quickly fine-tuned on new devices. It offers communication savings and privacy protection through partial local reconstruction and Top-P% selective communication, computational efficiency via online learning, and robustness to client heterogeneity through few-shot learning. The evaluations on three TinyML use cases demonstrate that TinyMetaFed can significantly reduce energy consumption and communication overhead, accelerate convergence, and stabilize the training process.
Open AccessTinyMetaFed: Efficient Federated Meta-Learning for TinyML
The field of Tiny Machine Learning (TinyML) has made substantial advancements in democratizing machine learning on low-footprint devices, such as microcontrollers. The prevalence of these miniature devices raises the question of whether aggregating their knowledge can benefit TinyML applications. Federated meta-learning is a promising answer to this question, as it addresses the scarcity of labeled data and heterogeneous data distribution across devices in the real world. However, deploying TinyML hardware faces unique resource constraints, making existing methods impractical due to energy, privacy, and communication limitations. We introduce TinyMetaFed, a model-agnostic meta-learning framework suitable for TinyML. TinyMetaFed facilitates collaborative training of a neural network initialization that can be quickly fine-tuned on new devices. It offers communication savings and privacy protection through partial local reconstruction and Top-P% selective communication, computational efficiency via online learning, and robustness to client heterogeneity through few-shot learning. The evaluations on three TinyML use cases demonstrate that TinyMetaFed can significantly reduce energy consumption and communication overhead, accelerate convergence, and stabilize the training process.
- Open Access
TinyReptile: TinyML with Federated Meta-Learning
Tiny machine learning (TinyML) is a rapidly growing field aiming to democratize machine learning (ML) for resource-constrained microcontrollers (MCUs). Given the pervasiveness of these tiny devices, it is inherent to ask whether TinyML applications can benefit from aggregating their knowledge. Federated learning (FL) enables decentralized agents to jointly learn a global model without sharing sensitive local data. However, a common global model may not work for all devices due to the complexity of the actual deployment environment and the heterogeneity of the data available on each device. In addition, the deployment of TinyML hardware has significant computational and communication constraints, which traditional ML fails to address. Considering these challenges, we propose TinyReptile, a simple but efficient algorithm inspired by meta-learning and online learning, to collaboratively learn a solid initialization for a neural network (NN) across tiny devices that can be quickly adapted to a new device with respect to its data. We demonstrate TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM. The evaluations on various TinyML use cases confirm a resource reduction and training time saving by at least two factors compared with baseline algorithms with comparable performance.
Open AccessTinyReptile: TinyML with Federated Meta-Learning
Tiny machine learning (TinyML) is a rapidly growing field aiming to democratize machine learning (ML) for resource-constrained microcontrollers (MCUs). Given the pervasiveness of these tiny devices, it is inherent to ask whether TinyML applications can benefit from aggregating their knowledge. Federated learning (FL) enables decentralized agents to jointly learn a global model without sharing sensitive local data. However, a common global model may not work for all devices due to the complexity of the actual deployment environment and the heterogeneity of the data available on each device. In addition, the deployment of TinyML hardware has significant computational and communication constraints, which traditional ML fails to address. Considering these challenges, we propose TinyReptile, a simple but efficient algorithm inspired by meta-learning and online learning, to collaboratively learn a solid initialization for a neural network (NN) across tiny devices that can be quickly adapted to a new device with respect to its data. We demonstrate TinyReptile on Raspberry Pi 4 and Cortex-M4 MCU with only 256-KB RAM. The evaluations on various TinyML use cases confirm a resource reduction and training time saving by at least two factors compared with baseline algorithms with comparable performance.
- Open Access
- 14 November 2022
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Internet Technology
- Vol. 22 (4) , 1-30
Towards Semantic Management of On-Device Applications in Industrial IoT
The Internet of Things (IoT) is revolutionizing the industry. Powered by pervasive embedded devices, the Industrial IoT (IIoT) provides a unique solution for retrieving and analyzing data near the source in real-time. Many emerging techniques, such as Tiny Machine Learning (TinyML) and Complex Event Processing (CEP) , are actively being developed to support decision making at the edge, shifting the paradigm from centralized processing to distributed computing. However, distributed computing presents management challenges, as IoT devices are diverse and constrained, and their number is growing exponentially. The situation is even more challenging when various on-device applications (so-called artifacts) are deployed across decentralized IoT networks. Questions to be addressed include how to discover an appropriate function, whether that function can be executed on a certain device, and how to orchestrate a cross-platform service. To tackle these challenges, we propose an approach for the scalable management of on-device applications among distributed IoT devices. By leveraging the W3C Web of Things (WoT) , the capabilities of each IoT device, or more precisely, its interaction patterns, can be semantically expressed in a Thing Description (TD) . In addition, we introduce semantic modeling of on-device applications to supplement an TD with additional information regarding applications on the device. Specifically, we demonstrate two examples of semantic modeling: neural networks (NN) and CEP rules. The ontologies are evaluated by answering a set of competency questions. By hosting the enriched semantic knowledge of the entire IoT system in a Knowledge Graph (KG) , we can discover and interoperate edge devices and artifacts across the decentralized network. This can reduce fragmentation and increase the reusability of IoT components. We demonstrate the feasibility of our concept on an industrial workstation consisting of a conveyor belt and several IoT devices. Finally, the requirements for constructing an IoT semantic management system are discussed.
- 14 November 2022
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Internet Technology
- Vol. 22 (4) , 1-30
Open AccessTowards Semantic Management of On-Device Applications in Industrial IoT
The Internet of Things (IoT) is revolutionizing the industry. Powered by pervasive embedded devices, the Industrial IoT (IIoT) provides a unique solution for retrieving and analyzing data near the source in real-time. Many emerging techniques, such as Tiny Machine Learning (TinyML) and Complex Event Processing (CEP) , are actively being developed to support decision making at the edge, shifting the paradigm from centralized processing to distributed computing. However, distributed computing presents management challenges, as IoT devices are diverse and constrained, and their number is growing exponentially. The situation is even more challenging when various on-device applications (so-called artifacts) are deployed across decentralized IoT networks. Questions to be addressed include how to discover an appropriate function, whether that function can be executed on a certain device, and how to orchestrate a cross-platform service. To tackle these challenges, we propose an approach for the scalable management of on-device applications among distributed IoT devices. By leveraging the W3C Web of Things (WoT) , the capabilities of each IoT device, or more precisely, its interaction patterns, can be semantically expressed in a Thing Description (TD) . In addition, we introduce semantic modeling of on-device applications to supplement an TD with additional information regarding applications on the device. Specifically, we demonstrate two examples of semantic modeling: neural networks (NN) and CEP rules. The ontologies are evaluated by answering a set of competency questions. By hosting the enriched semantic knowledge of the entire IoT system in a Knowledge Graph (KG) , we can discover and interoperate edge devices and artifacts across the decentralized network. This can reduce fragmentation and increase the reusability of IoT components. We demonstrate the feasibility of our concept on an industrial workstation consisting of a conveyor belt and several IoT devices. Finally, the requirements for constructing an IoT semantic management system are discussed.
- 16 October 2022
- Published by Springer Nature
- p. 845-862
SeLoC-ML: Semantic Low-Code Engineering for Machine Learning Applications in Industrial IoT
- 16 October 2022
- Published by Springer Nature
- p. 845-862
SeLoC-ML: Semantic Low-Code Engineering for Machine Learning Applications in Industrial IoT
- Open Access
TinyOL: TinyML with Online-Learning on Microcontrollers
Tiny machine learning (TinyML) is a fast-growing research area committed to democratizing deep learning for all-pervasive microcontrollers (MCUs). Challenged by the constraints on power, memory, and computation, TinyML has achieved significant advancement in the last few years. However, the current TinyML solutions are based on batch/offline settings and support only the neural network's inference on MCUs. The neural network is first trained using a large amount of pre-collected data on a powerful machine and then flashed to MCUs. This results in a static model, hard to adapt to new data, and impossible to adjust for different scenarios, which impedes the flexibility of the Internet of Things (IoT). To address these problems, we propose a novel system called TinyOL (TinyML with Online-Learning), which enables incremental on-device training on streaming data. TinyOL is based on the concept of online learning and is suitable for constrained IoT devices. We experiment TinyOL under supervised and unsupervised setups using an autoencoder neural network. Finally, we report the performance of the proposed solution and show its effectiveness and feasibility.
Open AccessTinyOL: TinyML with Online-Learning on Microcontrollers
Tiny machine learning (TinyML) is a fast-growing research area committed to democratizing deep learning for all-pervasive microcontrollers (MCUs). Challenged by the constraints on power, memory, and computation, TinyML has achieved significant advancement in the last few years. However, the current TinyML solutions are based on batch/offline settings and support only the neural network's inference on MCUs. The neural network is first trained using a large amount of pre-collected data on a powerful machine and then flashed to MCUs. This results in a static model, hard to adapt to new data, and impossible to adjust for different scenarios, which impedes the flexibility of the Internet of Things (IoT). To address these problems, we propose a novel system called TinyOL (TinyML with Online-Learning), which enables incremental on-device training on streaming data. TinyOL is based on the concept of online learning and is suitable for constrained IoT devices. We experiment TinyOL under supervised and unsupervised setups using an autoencoder neural network. Finally, we report the performance of the proposed solution and show its effectiveness and feasibility.