Exploring the Viability of Bypassing the Image Signal Processor for CNN-Based Object Detection in Autonomous Vehicles
Open Access
- 28 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 42302-42313
Abstract
In the field of autonomous driving, cameras are crucial sensors for providing information about a vehicle’s environment. Image quality refers to a camera system’s ability to capture, process, and display signals to form an image. Historically, "good quality" in this context refers to images that have been processed by an Image Signal Processor (ISP) designed with the goal of providing the optimal experience for human consumption. However, image quality perceived by humans may not always result in optimal conditions for computer vision. In the context of human consumption, image quality is well documented and understood. Image quality for computer vision applications, such as those in the autonomous vehicle industry, requires more research. Fully autonomous vehicles inevitably encounter constraints concerning data storage, transmission speed, and energy consumption. This is a result of enormous amounts of data being generated by the vehicle from suites made up of multiple different sensors. We propose a potential optimization along the computer vision pipeline, by completely bypassing the ISP block for a class of applications. We demonstrate that doing so has a negligible impact on the performance of Convolutional Neural Network (CNN) object detectors. The results also highlight the benefits of using raw pre-ISP data, in the context of computation and energy savings achieved by removing the ISP.Keywords
Funding Information
- Science Foundation Ireland (13/RC/2094)
- Southern and Eastern Regional Operational Program to Lero-the Science Foundation Ireland Research Centre for Software
This publication has 29 references indexed in Scilit:
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving2017
- Face Detection with the Faster R-CNN2017
- Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and DetectionIEEE Transactions on Medical Imaging, 2017
- You Only Look Once: Unified, Real-Time Object Detection2016
- On the Use of Feature Descriptors on Raw Image Data2016
- Fast R-CNN2015
- Microsoft COCO: Common Objects in Context2014
- Speeded-Up Robust Features (SURF)Computer Vision and Image Understanding, 2008
- Object recognition from local scale-invariant features1999