Uncertainty-Aware Projection Network for Multi-View 3D Object Detection
- 18 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (7) , 9278-9290
Abstract
In recent years, multi-view 3D object detection has drawn a lot of interest in autonomous driving. The projection from 2D image point to 3D world space is the core step of multi-view 3D object detection. However, the uncertainty of depth estimation and camera parameters can make the projected 3D position inaccurate, leading to a performance drop. To this end, we propose an Uncertainty-aware Projection Network (UPN), including a depth uncertainty-guided 3D point generation module and a camera-aware 3D position encoder. For the depth uncertainty-guided 3D point generation module, we predict the depth as a distribution to generate depth uncertainty-aware 3D points. For the camera-aware 3D position encoder, we alleviate the uncertainty of camera parameters by employing adaptive receptive fields to refine the position embedding. Based on these two modules, the proposed UPN can effectively deal with the performance drop caused by inaccurate 2D-3D projection. To the best of our knowledge, this is the first work to model the 2D-3D projection uncertainty for multi-view 3D object detection. The proposed UPN achieves 56.9% mAP and 65.4 % NDS on the nuScenes test dataset, demonstrating the superiority of our method.Keywords
Funding Information
- National Natural Science Foundation of China (42595545)
- Key Technology Research Project of TW-3 (TW3003)
- Youth Innovation Promotion Association of CAS
This publication has 62 references indexed in Scilit:
- OPEN: Object-Wise Position Embedding for Multi-view 3D Object Detection2024
- PAI3D: Painting Adaptive Instance-Prior for 3D Object Detection2023
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- PETR: Position Embedding Transformation for Multi-view 3D Object Detection2022
- RTM3D: Real-Time Monocular 3D Detection from Object Keypoints for Autonomous Driving2020
- Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D2020
- End-to-End Object Detection with Transformers2020
- Deeper Depth Prediction with Fully Convolutional Residual Networks2016
- ImageNet: A large-scale hierarchical image database2009
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955