MultiFusionNet: Spatiotemporal Camera–Radar Fusion in Dynamic Urban Environments

Authors:
Publisher website
Google Scholar
Abstract
As autonomous vehicles become more prevalent in urban settings, the need for accurate perception systems capable of high-precision localization and object detection in complex urban environments becomes increasingly critical. Traditional methods often struggle in underground structures or densely built urban zones where sensor data is obstructed or noisy. To address these challenges, we propose a novel integration of multiple cameras and radars, leveraging their complementary strengths to enhance overall perception accuracy. This approach moves beyond the limitations of single-sensor systems, combining the detailed visual data from cameras with the robust distance and velocity measurements from radars. Our innovative framework, MultiFusionNet, incorporates deep learning, adaptive multimodal fusion, and multiscale attention mechanisms to dynamically adjust to varying environmental conditions and complex sensor data interactions. Additionally, we introduce the scene-aware spatiotemporal network (SA-STNet) to refine radar data and improve its resolution and temporal coherence. By optimizing sensor placement with an uncertainty-aware probabilistic metric, we achieve comprehensive environmental coverage and significantly enhance the reliability of autonomous navigation. By optimizing sensor placement with an uncertainty-aware probabilistic metric, we achieve comprehensive environmental coverage, leading to improved detection accuracy (e.g., 0.88 AP and 0.90 recall) in simulated urban scenarios, paving the way for enhanced autonomous navigation reliability pending real-world validation. Validated through extensive simulations in the CARLA-generated datasets, our framework demonstrates notable improvements in detection accuracy, offering a robust solution for navigating complex and challenging urban landscapes.

This publication has 33 references indexed in Scilit: