Monocular Plan View Networks for Autonomous Driving
Authors:
- Dequan WangDWDequan Wang
- Coline DevinCDColine Devin
- Qi-Zhi CaiQCQi-Zhi Cai
- Philipp KrähenbühlPKPhilipp Krähenbühl
- Trevor DarrellTDTrevor Darrell
Preprint
Abstract
Convolutions on monocular dash cam videos capture spatial invariances in the image plane but do not explicitly reason about distances and depth. We propose a simple transformation of observations into a bird's eye view, also known as plan view, for end-to-end control. We detect vehicles and pedestrians in the first person view and project them into an overhead plan view. This representation provides an abstraction of the environment from which a deep network can easily deduce the positions and directions of entities. Additionally, the plan view enables us to leverage advances in 3D object detection in conjunction with deep policy learning. We evaluate our monocular plan view network on the photo-realistic Grand Theft Auto V simulator. A network using both a plan view and front view causes less than half as many collisions as previous detection-based methods and an order of magnitude fewer collisions than pure pixel-based policies.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- On Offline Evaluation of Vision-Based Driving Models2018
- Deep Continuous Fusion for Multi-sensor 3D Object Detection2018
- Efficient Convolutions for Real-Time Semantic Segmentation of 3D Point Clouds2018
- Learning to Map Vehicles into Bird’s Eye View2017
- End-to-End Learning of Driving Models from Large-Scale Video Datasets2017
- Monocular 3D Object Detection for Autonomous Driving2016
- A coarse-to-fine model for 3D pose estimation and sub-category recognition2015
- Microsoft COCO: Common Objects in Context2014
- OpenDR: An Approximate Differentiable Renderer2014
- Are we ready for autonomous driving? The KITTI vision benchmark suite2012