BEVSOC: Self-Supervised Contrastive Learning for Calibration-Free BEV 3-D Object Detection
- 20 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (12) , 22167-22182
Abstract
3D object detection based on multi-view cameras and bird’s-eye view (BEV) representation is a key task for autonomous driving, as it enables the perception systems to understand the surrounding scenes. However, most existing BEV representation methods rely on the projection matrix of camera intrinsic and extrinsic parameters, which requires a complex and time-consuming calibration process that may introduce errors and degrade the detection performance. Moreover, the calibration results may vary due to environmental changes and affect the stability of the detection system. To address this problem, we propose a calibration-free 3D object detection method that leverages a group-equivariant convolutional network to extract features from multi-view images and a projection network module to learn the implicit 3D-to-2D projection relationship for obtaining BEV representation. Furthermore, we employ contrastive learning to pre-train the projection network module without using manually annotated data. By exploiting the multi-view camera data through contrastive learning, our proposed method eliminates the need for tedious calibration, avoids calibration errors, and reduces the dependence on a large amount of annotated data for calibration-free 3D object detection. We evaluate our method on the nuScenes dataset and demonstrate its competitive performance. Our method improves the stability and reliability of 3D object detection in long-term autonomous driving.Keywords
Funding Information
- Key Research and Development Project of Hainan Province (ZDYF2022GXJS348, ZDYF2022SHFZ039)
- National Natural Science Foundation of China (62161010, 62361024)
- Hainan Seed Industry Laboratory (B23H10004)
This publication has 41 references indexed in Scilit:
- AGAM-SLAM: An Adaptive Dynamic Scene Semantic SLAM Method Based on GAM2023
- ATY-SLAM: A Visual Semantic SLAM for Dynamic Indoor Environments2023
- GET: group equivariant transformer for person detection of overhead fisheye imagesApplied Intelligence, 2023
- BEVFormer: Learning Bird’s-Eye-View Representation from Multi-camera Images via Spatiotemporal Transformers2022
- Calibration-Free Multi-view Crowd Counting2022
- Learning Ego 3D Representation as Ray Tracing2022
- PETR: Position Embedding Transformation for Multi-view 3D Object Detection2022
- Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D2020
- Multiview Detection with Feature Perspective Transformation2020
- End-to-End Object Detection with Transformers2020