MV-Map: Offboard HD-Map Generation with Multi-view Consistency
Preprint
Abstract
While bird's-eye-view (BEV) perception models can be useful for building high-definition maps (HD-Maps) with less human labor, their results are often unreliable and demonstrate noticeable inconsistencies in the predicted HD-Maps from different viewpoints. This is because BEV perception is typically set up in an 'onboard' manner, which restricts the computation and consequently prevents algorithms from reasoning multiple views simultaneously. This paper overcomes these limitations and advocates a more practical 'offboard' HD-Map generation setup that removes the computation constraints, based on the fact that HD-Maps are commonly reusable infrastructures built offline in data centers. To this end, we propose a novel offboard pipeline called MV-Map that capitalizes multi-view consistency and can handle an arbitrary number of frames with the key design of a 'region-centric' framework. In MV-Map, the target HD-Maps are created by aggregating all the frames of onboard predictions, weighted by the confidence scores assigned by an 'uncertainty network'. To further enhance multi-view consistency, we augment the uncertainty network with the global 3D structure optimized by a voxelized neural radiance field (Voxel-NeRF). Extensive experiments on nuScenes show that our MV-Map significantly improves the quality of HD-Maps, further highlighting the importance of offboard methods for HD-Map generation.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- MOTR: End-to-End Multiple-Object Tracking with Transformer2022
- Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction2022
- Point-NeRF: Point-based Neural Radiance Fields2022
- Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D2020
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- Argoverse: 3D Tracking and Forecasting With Rich Maps2019
- PointPillars: Fast Encoders for Object Detection From Point Clouds2019
- MVSNet: Depth Inference for Unstructured Multi-view Stereo2018
- Deep Residual Learning for Image Recognition2016