Analysis of Neural Network Inference Response Times on Embedded Platforms
Authors:
- 24 October 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-7
Abstract
The response time of Artificial Neural Network (ANN)-inference is of utmost importance in embedded applications, particularly continual stream-processing. Predictive maintenance applications require timely predictions of state changes. This study serves to enable the reader to estimate the response time of a given model based on the underlying platform, and emphasizes the relevance of benchmarking generic ANN applications on edge devices. We analyze the influence of net parameters, activation functions as well as single-and multithreading on execution times. Potential side effects such as tact rate variances or other hardware-related influences are being outlined and accounted for. The results underline the complexity of task-partitioning and scheduling strategies while emphasizing the necessity of precise concertation of the parameters to achieve optimal performance on any platform. This study shows that cutting-edge frameworks don’t necessarily perform the required concertations automatically for all configurations, which may negatively impact performance.Keywords
This publication has 8 references indexed in Scilit:
- Deep neural network-based secure healthcare frameworkNeural Computing and Applications, 2024
- DeepEdgeBench: Benchmarking Deep Neural Networks on Edge Devices2021
- Performance Evaluation of Deep Learning Compilers for Edge Inference2021
- Increasing Neurons or Deepening Layers in Forecasting Maximum Temperature Time Series?Atmosphere, 2020
- Superiority of Simplicity: A Lightweight Model for Network Device Workload PredictionAnnals of Computer Science and Information Systems, 2020
- Fully Convolutional Deep Neural Networks with Optimized Hyperparameters for Detection of Shockable and Non-Shockable RhythmsSensors, 2020
- Predictive Maintenance of Power Substation Equipment by Infrared Thermography Using a Machine-Learning ApproachEnergies, 2017
- Fathom: reference workloads for modern deep learning methods2016