Increased Leverage of Transprecision Computing for Machine Vision Applications at the Edge
Authors:
Open Access
- 30 June 2022
- journal article
- research article
- Published by Springer Nature in Journal of Signal Processing Systems
- Vol. 94 (10) , 1101-1118
Abstract
The practical deployment of machine vision presents particular challenges for resource constrained edge devices. With a clear need to execute multiple tasks with variable workloads, there is a need for a robust approach that can dynamically adapt at runtime and which can maintain the maximum quality of service (QoS) within the available resource constraints. A lightweight approach that monitors the runtime workload constraints and leverages accuracy-throughput trade-offs on a graphics processing unit (GPU), is presented. It includes optimisation techniques that identify the configurations for each task in terms of optimal accuracy, energy and memory and management of the transparent switching between configurations. Using a neural network architecture search that statically generates a range of implementations that target a resource-precision trade-off, we explore the detection of the optimal parameters for the required QoS under specific memory and energy constraints. For an accuracy loss of 1%, we demonstrate that a $$1.6\times$$ higher frame processing rate can be achieved on GPU with further improvements possible at further relaxed accuracy. In order to further improve the switching between configurations, we enhance the proposed mechanism by employing central processing units (CPUs) for offloading some of the executed frames, which helps to improve the frame rate by further 0.9%.
Keywords
Funding Information
- CHIST-ERA (CHIST-ERA-18-SDCDN-00)
- Engineering and Physical Sciences Research Council (EP/T022345/1)
This publication has 37 references indexed in Scilit:
- Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices2017
- Accelerating Deep Convolutional Networks using low-precision and sparsity2017
- BranchyNet: Fast inference via early exiting from deep neural networks2016
- Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition2016
- Channel-aware energy optimization of OFDM receivers using dynamic precision scaling in FPGAS2015
- Going deeper with convolutions2015
- Opportunities for Energy Efficient Computing: A Study of Inexact General Purpose Processors for High-Performance and Big-Data Applications2015
- OpenTuner2014
- Fine grain word length optimization for dynamic precision scaling in DSP systems2013
- Logic and Memory Design Based on Unequal Error Protection for Voltage-scalable, Robust and Adaptive DSP SystemsJournal of Signal Processing Systems, 2011