SMM-POD: Panoramic 3D Object Detection via Spherical Multi-Stage Multi-Modal Fusion
Open Access
- 18 June 2025
- journal article
- research article
- Published by MDPI AG in Remote Sensing
- Vol. 17 (12) , 2089
Abstract
Panoramic 3D object detection is a challenging task due to image distortion, sensor heterogeneity, and the difficulty of combining information from multiple modalities over a wide field-of-view (FoV). To address these issues, we propose SMM-POD, a novel framework that introduces a spherical multi-stage fusion strategy for panoramic 3D detection. Our approach creates a five-channel spherical image aligned with LiDAR data and uses a quasi-uniform Voronoi sphere (UVS) model to reduce projection distortion. A cross-attention-based feature extraction module and a transformer encoder–decoder with spherical positional encoding enable the accurate and efficient fusion of image and point cloud features. For precise 3D localization, we adopt a Frustum PointNet module. Experiments on the DAIR-V2X-I benchmark and our self-collected SHU-3DPOD dataset show that SMM-POD achieves a state-of-the-art performance across all object categories. It significantly improves the detection of small objects like cyclists and pedestrians and maintains stable results under various environmental conditions. These results demonstrate the effectiveness of SMM-POD in panoramic multi-modal 3D perception and establish it as a strong baseline for wide FoV object detection.Keywords
Funding Information
- National Natural Science Foundation of China (62173220, 62303294)
This publication has 45 references indexed in Scilit:
- SGV3D: Toward Scenario Generalization for Vision-Based Roadside 3D Object DetectionIEEE Transactions on Intelligent Transportation Systems, 2025
- CoBEV: Elevating Roadside 3D Object Detection With Depth and Height ComplementarityIEEE Transactions on Image Processing, 2024
- Occlusion-guided multi-modal fusion for vehicle-infrastructure cooperative 3D object detectionPattern Recognition, 2024
- MonoGAE: Roadside Monocular 3D Object Detection With Ground-Aware EmbeddingsIEEE Transactions on Intelligent Transportation Systems, 2024
- UVS-CNNs: Constructing general convolutional neural networks on quasi-uniform spherical imagesComputers & Graphics, 2024
- Multi-modality 3D object detection in autonomous driving: A reviewNeurocomputing, 2023
- PVF-DectNet: Multi-modal 3D detection network based on Perspective-Voxel fusionEngineering Applications of Artificial Intelligence, 2023
- LiDAR-camera fusion: Dual transformer enhancement for 3D object detectionEngineering Applications of Artificial Intelligence, 2023
- EPNet++: Cascade Bi-Directional Fusion for Multi-Modal 3D Object DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2022
- PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object DetectionInternational Journal of Computer Vision, 2022