Abstract
Continual Federated Learning (CFL) for 3D object classification requires learning generalizable representations across distributed clients while maintaining feature-space consistency as new classes emerge. However, existing CFL methods, which are primarily designed for 2D vision tasks, fail to address the geometric complexity of 3D point clouds and lack mechanisms to simultaneously mitigate intra-task and inter-task forgetting. To address these limitations, we propose DUAL3D-Fed, a dual-distillation framework that supports scalable, privacy-preserving, and semantically coherent 3D continual federated learning. First, we propose an embedding-space distillation, where we align local and global representation distributions using Sinkhorn divergence, a regularized optimal transport metric that enforces consistency across client models and over time. Next, we introduce semantic grounding withVision-Language Models (VLMs), which uses fixed language prompts to anchor the 3D feature space to a rich semantic prior. This promotes clustering of point cloud samples with similar textual descriptions and separation of semantically dissimilar ones. Finally, a Logit-space distillation is added, where we stabilize class predictions using a hybrid distillation scheme that incorporates buffered exemplars to retain historical knowledge while adapting to novel classes. Experiments on four synthetic and real-world 3D benchmarks under CFL settings show that DUAL3D-Fed improves classification accuracy, enhanced feature representations, and exhibits greater robustness to class drift and forgetting than baseline methods.
Funding Information
  • Commonwealth Scientific and Industrial Research Organisations (CSIRO’s) Data61 Science Digital, and in part by the CSIRO’s Data61 Embodied AI Cluster