In-Vehicle Object-Level 3D Reconstruction of Traffic Scenes
- 28 July 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 22 (12) , 7747-7759
Abstract
Emerging automotive applications such as in-vehicle Augmented Reality (AR) and fully automated parking require a comprehensive understanding of the vehicle’s three-dimensional surrounding represented as an object-level environmental model. In this model, not only 3D poses (positions and orientations) and 3D sizes of detected objects are registered, but 3D shapes (geometries) need to be reconstructed precisely. A combination of 3D object detection and 3D surface reconstruction techniques, referred to as object-level 3D reconstruction , is fundamental to building such environmental models. However, the possibilities to incorporate object-level 3D reconstruction in a car have not been sufficiently explored either in academic research or in the industry. This primarily stems from the cost and resource constraints associated with the automotive domain. In this paper, we address these constraints by proposing implementations of in-vehicle object-level 3D reconstruction in two specific use cases: (i) augmented reality and (ii) automated parking. For augmented reality, we propose a cost-efficient solution called monocular 3D Shaping that requires only a single frame from a monocular camera as input. For automated parking, we propose a resource-efficient alternative that generates more precise 3D reconstruction results by taking advantage of additional 3D sensors (such as Lidars). The crux of our proposed approaches lies in the use of a Latent Shape Space , where various 3D shapes are represented using only two parameters. As a result, highly complex 3D shapes can now be transmitted using a low- to medium-bandwidth in-vehicle communication infrastructure in a cost-effective manner.Keywords
This publication has 36 references indexed in Scilit:
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- PWP3D: Real-Time Segmentation and Tracking of 3D ObjectsInternational Journal of Computer Vision, 2012
- A Unified Energy Minimization Framework for Model Fitting in Depth2012
- SCAPEACM Transactions on Graphics, 2005
- Active Shape Models-Their Training and ApplicationComputer Vision and Image Understanding, 1995
- The visual hull concept for silhouette-based image understandingIEEE Transactions on Pattern Analysis and Machine Intelligence, 1994
- A scaled conjugate gradient algorithm for fast supervised learningNeural Networks, 1993
- On the geometric interpretation of image contoursArtificial Intelligence, 1988
- Marching cubes: A high resolution 3D surface construction algorithmACM SIGGRAPH Computer Graphics, 1987
- The Levenberg-Marquardt algorithm: Implementation and theory1978