DiffSemanticFusion: Semantic Raster BEV Fusion for Autonomous Driving via Online Map Diffusion
- 12 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 2354-2361
Abstract
Autonomous driving requires accurate scene understanding, including road geometry, traffic agents, and their semantic relationships. In online HD map generation scenarios, raster-based representations are well-suited to vision models but lack geometric precision, while graph-based representations retain structural detail but become unstable without precise maps. To harness the complementary strengths of both, we propose DiffSemanticFusion—a fusion framework for multimodal trajectory prediction and planning. Our approach reasons over a semantic raster–fused BEV space, enhanced by a map diffusion module that improves both the stability and expressiveness of online HD map representations. We validate our framework on two downstream tasks: trajectory prediction and planning-oriented end-to-end autonomous driving. Experiments on real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate improved performance over several state-of-the-art (SOTA) methods. For the prediction task on nuScenes, we integrate DiffSemanticFusion with the online HD map informed QCNet, achieving a 5.1% performance improvement. For end-to-end autonomous driving in NAVSIM, DiffSemanticFusion achieves SOTA results, with a 15% performance gain in NavHard scenarios. In addition, extensive ablation and sensitivity studies show that our map diffusion module can be seamlessly integrated into other vector-based approaches to enhance performance.Keywords
All Related Versions
Funding Information
- Bosch Corporate Research
This publication has 34 references indexed in Scilit:
- MapTRv2: An End-to-End Framework for Online Vectorized HD Map ConstructionInternational Journal of Computer Vision, 2024
- MOTR: End-to-End Multiple-Object Tracking with Transformer2022
- ByteTrack: Multi-object Tracking by Associating Every Detection Box2022
- HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction2022
- Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D2020
- Learning Lane Graph Representations for Motion Forecasting2020
- CoverNet: Multimodal Behavior Prediction Using Trajectory Sets2020
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation2020
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018