FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Model
Preprint
Abstract
3D semantic occupancy prediction plays a pivotal role in autonomous driving. However, inherent limitations of fewframe images and redundancy in 3D space compromise prediction accuracy for occluded and distant scenes. Existing methods enhance performance by fusing historical frame data, which need additional data and significant computational resources. To address these issues, this paper propose FMOcc, a Tri-perspective View (TPV) refinement occupancy network with flow matching selective state space model for few-frame 3D occupancy prediction. Firstly, to generate missing features, we designed a feature refinement module based on a flow matching model, which is called Flow Matching SSM module (FMSSM). Furthermore, by designing the TPV SSM layer and Plane Selective SSM (PS3M), we selectively filter TPV features to reduce the impact of air voxels on non-air voxels, thereby enhancing the overall efficiency of the model and prediction capability for distant scenes. Finally, we design the Mask Training (MT) method to enhance the robustness of FMOcc and address the issue of sensor data loss. Experimental results on the Occ3D-nuScenes and OpenOcc datasets show that our FMOcc outperforms existing state-of-theart methods. Our FMOcc with two frame input achieves notable scores of 43.1% RayIoU and 39.8% mIoU on Occ3D-nuScenes validation, 42.6% RayIoU on OpenOcc with 5.4 G inference memory and 330ms inference time.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- OccMamba: Semantic Occupancy Prediction with State Space Models2025
- LocalMamba: Visual State Space Model with Windowed Selective Scan2025
- GEOcc: Geometrically Enhanced 3D Occupancy Network With Implicit-Explicit Depth Fusion and Contextual Self-SupervisionIEEE Transactions on Intelligent Transportation Systems, 2025
- OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving2024
- Diffusion Model for Robust Multi-sensor Fusion in 3D Object Detection and BEV Segmentation2024
- Fully Sparse 3D Occupancy Prediction2024
- SuPrNet: Super Proxy for 4D occupancy forecastingKnowledge-Based Systems, 2024
- Autonomous mining through cooperative driving and operations enabled by parallel intelligenceCommunications Engineering, 2024
- Co-Occ: Coupling Explicit Feature Fusion With Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy PredictionIEEE Robotics and Automation Letters, 2024
- Road-Model-Based Road Boundary Extraction for High Definition Map via LIDARIEEE Transactions on Intelligent Transportation Systems, 2022