Optimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach
Open Access
- 16 July 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
Abstract
The use of hyperspectral imaging (HSI) for autonomous navigation is a promising field of research that aims to improve the accuracy and robustness of detection, tracking, and scene understanding systems based on vision sensors. The combination of advanced computer algorithms, such as deep neural networks (DNNs), and small-size snapshot HSI cameras allows to strengthen the reliability of those vision systems. Using HSI, some intrinsic limitations of greyscale and RGB imaging in depicting physical properties of targets related to the spectral reflectance of materials (metamerism) are overcome. Despite the promising results of many published HSI-based computer vision developments, the strict requirements of safety-critical applications such as autonomous driving systems (ADS) regarding latency, resource consumption, and security are prompting the migration of machine learning (ML)-based solutions to edge platforms. This involves a thorough software/hardware co-design scheme to distribute and optimize the tasks efficiently among the limited resources of computing platforms. With respect to inference, the over-parameterized nature of DNNs poses significant computational challenges for real-time on-the-edge deployment. In addition, the intensive data preprocessing required by HSI, which is frequently overlooked, must be carefully managed in terms of memory arrangement and inter-task communication to enable an efficient integrated pipeline design on a system on chip (SoC). This work presents a set of optimization techniques for the practical co-design of a DNN-based HSI segmentation processor deployed on a field programmable gate array (FPGA)-based SoC targeted at ADS, including key optimizations such as functional software/hardware task distribution, hardware-aware preprocessing, ML model compression, and a complete pipelined deployment. Applied compression techniques significantly reduce the complexity of the designed DNN to 24.34% of the original operations and to 1.02% of the original number of parameters, achieving a 2.86x speed-up in the inference task without noticeable degradation of the segmentation accuracy.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- A comprehensive review of network pruning based on pruning granularity and pruning time perspectivesNeurocomputing, 2025
- Evaluating single event upsets in deep neural networks for semantic segmentation: An embedded system perspectiveJournal of Systems Architecture, 2024
- On-chip hyperspectral image segmentation with fully convolutional networks for scene understanding in autonomous drivingJournal of Systems Architecture, 2023
- Auto-Compressing Subset Pruning for Semantic Image Segmentation2022
- STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentationMedical Image Analysis, 2022
- Review the state-of-the-art technologies of semantic segmentation based on deep learningNeurocomputing, 2022
- A dynamic CNN pruning method based on matrix similaritySignal, Image and Video Processing, 2020
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Semantic object classes in video: A high-definition ground truth databasePattern Recognition Letters, 2009