3D-FAD: Informativeness-guided Multimodal 3D Object Detection for Federated Autonomous Driving
Open Access
- 9 July 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Sensor Networks
- Vol. 22 (4) , 1-29
Abstract
Autonomous driving has been adopted in various applications, including taxis, logistics, and agriculture, to address labor shortages and enhance safety. A critical component is 3D object detection, relying on deep neural networks (DNNs) that integrate LiDAR and camera data for robust road planning. Federated Autonomous Driving (FAD) utilizes Federated Learning (FL) to train these DNNs, addressing the demand for diverse training datasets while preserving client privacy. However, extending the existing FAD framework to 3D object detection remains challenging due to insufficient consideration of modality heterogeneity and informativeness. The article introduces 3D-FAD , a pioneering informativeness-guided multimodal FAD framework. 3D-FAD is the first to explicitly integrate modality informativeness into multimodal 3D object detection for FAD by a convolution-based encoder. It utilizes this informativeness for efficient client selection and aggregation at the server. Furthermore, 3D-FAD introduces Masked Modality AutoEncoders to impute deficient LiDAR features for clients with limited LiDAR FOV and low-cost cameras in real-world autonomous driving. Our experiments on real-world autonomous driving datasets nuScene and Waymo show that 3D-FAD outperforms baselines by up to 34.1% and 39.2% in key metrics, along with better performance across different object classes and environments.Keywords
Funding Information
- NSF (NeTS-2312674)
- NSF (CNS-2312448)
This publication has 65 references indexed in Scilit:
- An active federated method driven by inter-client informativeness variability of labeled dataSignal, Image and Video Processing, 2023
- Multi-Modal 3D Object Detection in Autonomous Driving: A SurveyInternational Journal of Computer Vision, 2023
- A survey on federated learningKnowledge-Based Systems, 2021
- Multi-Modal Stacked Denoising Autoencoder for Handling Missing Data in Healthcare Big DataIEEE Access, 2020
- Deep Continuous Fusion for Multi-sensor 3D Object Detection2018
- Frustum PointNets for 3D Object Detection from RGB-D Data2018
- PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation2018
- Multi-view 3D Object Detection Network for Autonomous Driving2017
- Missing Modalities Imputation via Cascaded Residual Autoencoder2017
- Context Encoders: Feature Learning by Inpainting2016