Rafael Stahl
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Rafael Stahl has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Ulf Schlichtmann
Technical University of Munich, Munich, Bavaria, Germany
6 shared publications - Daniel Mueller-Gritschneder
TU Wien
4 shared publications Daniel Mueller-Gritschneder
Chair of Electronic Design Automation Technical University of Munich Munich Germany
3 shared publications- Philipp van Kempen
Technical University of Munich, Munich, Germany
2 shared publications - Alexander Hoffman
Technical University of Munich, Munich, Germany
1 shared publication Andreas Gerstlauer
University of Texas at Austin
1 shared publication
Latest Publications
- Open Access
- 21 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 32-36
MLonMCU: TinyML Benchmarking with Fast Retargeting
While there exist many ways to deploy machine learning models on microcontrollers, it is non-trivial to choose the optimal combination of frameworks and targets for a given application. Thus, automating the end-to-end benchmarking flow is of high relevance nowadays. A tool called MLonMCU is proposed in this paper and demonstrated by benchmarking the state-of-the-art TinyML frameworks TFLite for Microcontrollers and TVM effortlessly with a large number of configurations in a low amount of time.
- 21 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 32-36
Open AccessMLonMCU: TinyML Benchmarking with Fast Retargeting
While there exist many ways to deploy machine learning models on microcontrollers, it is non-trivial to choose the optimal combination of frameworks and targets for a given application. Thus, automating the end-to-end benchmarking flow is of high relevance nowadays. A tool called MLonMCU is proposed in this paper and demonstrated by benchmarking the state-of-the-art TinyML frameworks TFLite for Microcontrollers and TVM effortlessly with a large number of configurations in a low amount of time.
- Open Access
- 21 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 6-10
Towards Rapid Exploration of Heterogeneous TinyML Systems using Virtual Platforms and TVM's UMA
The rapid setup of deep learning compilation toolchains for heterogeneous TinyML systems with a processor and dedicated ML accelerator is still at an early stage. Here, achieving the most optimal combination of targets for a TinyML application on ultra-low-power edge devices demands additional benchmarking solutions to estimate the final performance. Apache TVM's Universal Modular Accelerator (UMA) interface as an easy-to-use API is a promising speed-up approach to this scope. In this paper, we integrate a simple custom dedicated accelerator into TVM using UMA to offload the quantized convolution operators in order to demonstrate such an approach. Furthermore, we leverage MLonMCU tool and its capability of virtual prototyping to estimate and explore the performance improvement achieved by the accelerator.
- 21 September 2023
- Published by Association for Computing Machinery (ACM)
- p. 6-10
Open AccessTowards Rapid Exploration of Heterogeneous TinyML Systems using Virtual Platforms and TVM's UMA
The rapid setup of deep learning compilation toolchains for heterogeneous TinyML systems with a processor and dedicated ML accelerator is still at an early stage. Here, achieving the most optimal combination of targets for a TinyML application on ultra-low-power edge devices demands additional benchmarking solutions to estimate the final performance. Apache TVM's Universal Modular Accelerator (UMA) interface as an easy-to-use API is a promising speed-up approach to this scope. In this paper, we integrate a simple custom dedicated accelerator into TVM using UMA to offload the quantized convolution operators in order to demonstrate such an approach. Furthermore, we leverage MLonMCU tool and its capability of virtual prototyping to estimate and explore the performance improvement achieved by the accelerator.
- Open Access
MLonMCU: TinyML Benchmarking with Fast Retargeting
While there exist many ways to deploy machine learning models on microcontrollers, it is non-trivial to choose the optimal combination of frameworks and targets for a given application. Thus, automating the end-to-end benchmarking flow is of high relevance nowadays. A tool called MLonMCU is proposed in this paper and demonstrated by benchmarking the state-of-the-art TinyML frameworks TFLite for Microcontrollers and TVM effortlessly with a large number of configurations in a low amount of time.
Open AccessMLonMCU: TinyML Benchmarking with Fast Retargeting
While there exist many ways to deploy machine learning models on microcontrollers, it is non-trivial to choose the optimal combination of frameworks and targets for a given application. Thus, automating the end-to-end benchmarking flow is of high relevance nowadays. A tool called MLonMCU is proposed in this paper and demonstrated by benchmarking the state-of-the-art TinyML frameworks TFLite for Microcontrollers and TVM effortlessly with a large number of configurations in a low amount of time.
- Open Access
Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Memory optimization for deep neural network (DNN) inference gains high relevance with the emergence of TinyML, which refers to the deployment of DNN inference tasks on tiny, low-power microcontrollers. Applications such as audio keyword detection or radar-based gesture recognition are heavily constrained by the limited memory on such tiny devices because DNN inference requires large intermediate run-time buffers to store activations and other intermediate data, which leads to high memory usage. In this paper, we propose a new Fused Depthwise Tiling (FDT) method for the memory optimization of DNNs, which, compared to existing tiling methods, reduces memory usage without inducing any run time overhead. FDT applies to a larger variety of network layers than existing tiling methods that focus on convolutions. It improves TinyML memory optimization significantly by reducing memory of models where this was not possible before and additionally providing alternative design points for models that show high run time overhead with existing methods. In order to identify the best tiling configuration, an end-to-end flow with a new path discovery method is proposed, which applies FDT and existing tiling methods in a fully automated way, including the scheduling of the operations and planning of the layout of buffers in memory. Out of seven evaluated models, FDT achieved significant memory reduction for two models by 76.2% and 18.1% where existing tiling methods could not be applied. Two other models showed a significant run time overhead with existing methods and FDT provided alternative design points with no overhead but reduced memory savings.
Open AccessFused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Memory optimization for deep neural network (DNN) inference gains high relevance with the emergence of TinyML, which refers to the deployment of DNN inference tasks on tiny, low-power microcontrollers. Applications such as audio keyword detection or radar-based gesture recognition are heavily constrained by the limited memory on such tiny devices because DNN inference requires large intermediate run-time buffers to store activations and other intermediate data, which leads to high memory usage. In this paper, we propose a new Fused Depthwise Tiling (FDT) method for the memory optimization of DNNs, which, compared to existing tiling methods, reduces memory usage without inducing any run time overhead. FDT applies to a larger variety of network layers than existing tiling methods that focus on convolutions. It improves TinyML memory optimization significantly by reducing memory of models where this was not possible before and additionally providing alternative design points for models that show high run time overhead with existing methods. In order to identify the best tiling configuration, an end-to-end flow with a new path discovery method is proposed, which applies FDT and existing tiling methods in a fully automated way, including the scheduling of the operations and planning of the layout of buffers in memory. Out of seven evaluated models, FDT achieved significant memory reduction for two models by 76.2% and 18.1% where existing tiling methods could not be applied. Two other models showed a significant run time overhead with existing methods and FDT provided alternative design points with no overhead but reduced memory savings.
- Open Access
- 7 April 2021
- Published by Springer Nature in International Journal of Parallel Programming
- Vol. 49 (4) , 600-624
DeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices
Performing inference of Convolutional Neural Networks (CNNs) on Internet of Things (IoT) edge devices ensures both privacy of input data and possible run time reductions when compared to a cloud solution. As most edge devices are memory- and compute-constrained, they cannot store and execute complex CNNs. Partitioning and distributing layer information across multiple edge devices to reduce the amount of computation and data on each device presents a solution to this problem. In this article, we propose DeeperThings, an approach that supports a full distribution of CNN inference tasks by partitioning fully-connected as well as both feature- and weight-intensive convolutional layers. Additionally, we jointly optimize memory, computation and communication demands. This is achieved using techniques to combine both feature and weight partitioning with a communication-aware layer fusion method, enabling holistic optimization across layers. For a given number of edge devices, the schemes are applied jointly using Integer Linear Programming (ILP) formulations to minimize data exchanged between devices, to optimize run times and to find the entire model’s minimal memory footprint. Experimental results from a real-world hardware setup running four different CNN models confirm that the scheme is able to evenly balance the memory footprint between devices. For six devices on 100 Mbit/s connections the integration of layer fusion additionally leads to a reduction of communication demands by up to 28.8%. This results in run time speed-up of the inference task by up to 1.52x compared to layer partitioning without fusing.
- 7 April 2021
- Published by Springer Nature in International Journal of Parallel Programming
- Vol. 49 (4) , 600-624
Open AccessDeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices
Performing inference of Convolutional Neural Networks (CNNs) on Internet of Things (IoT) edge devices ensures both privacy of input data and possible run time reductions when compared to a cloud solution. As most edge devices are memory- and compute-constrained, they cannot store and execute complex CNNs. Partitioning and distributing layer information across multiple edge devices to reduce the amount of computation and data on each device presents a solution to this problem. In this article, we propose DeeperThings, an approach that supports a full distribution of CNN inference tasks by partitioning fully-connected as well as both feature- and weight-intensive convolutional layers. Additionally, we jointly optimize memory, computation and communication demands. This is achieved using techniques to combine both feature and weight partitioning with a communication-aware layer fusion method, enabling holistic optimization across layers. For a given number of edge devices, the schemes are applied jointly using Integer Linear Programming (ILP) formulations to minimize data exchanged between devices, to optimize run times and to find the entire model’s minimal memory footprint. Experimental results from a real-world hardware setup running four different CNN models confirm that the scheme is able to evenly balance the memory footprint between devices. For six devices on 100 Mbit/s connections the integration of layer fusion additionally leads to a reduction of communication demands by up to 28.8%. This results in run time speed-up of the inference task by up to 1.52x compared to layer partitioning without fusing.
- 18 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 12 (2) , 66-69
Driver Generation for IoT Nodes With Optimization of the Hardware/Software Interface
- 18 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 12 (2) , 66-69
Driver Generation for IoT Nodes With Optimization of the Hardware/Software Interface
- 8 August 2019
- Published by Springer Nature
- p. 77-90
Fully Distributed Deep Learning Inference on Resource-Constrained Edge Devices
- 8 August 2019
- Published by Springer Nature
- p. 77-90
Fully Distributed Deep Learning Inference on Resource-Constrained Edge Devices