Dongdong Li

Integrating neural rendering with simultaneous localization and mapping (SLAM) has shown great promise for achieving high-precision localization and photorealistic scene reconstruction. However, the dynamic perspectives of uncrewed aerial vehicles (UAVs), compounded by Global Navigation Satellite System (GNSS) signal interference and the accumulation of localization errors, pose significant challenges to existing monocular SLAM methods in complex environments. To address these limitations, we propose LD3DGS-SLAM, a novel framework that integrates monocular SLAM, GNSS, and 3-D Gaussian splatting (3DGS) to enhance UAV-based localization and mapping. The system integrates traditional geometric feature-based localization with the multiview synthesis capability of 3-D Gaussian rendering, achieving a breakthrough in accuracy while enhancing robustness. First, we construct a multisensor fusion graph optimization model that tightly integrates GNSS and monocular vision data, effectively mitigating the cumulative drift typical of traditional SLAM pipelines. Afterward, we introduce a 3-D Gaussian-based mapping strategy that incrementally refines a dense scene representation using SLAM-generated point clouds, fusing geometric and texture information to reconstruct highly accurate and detailed maps. To further enhance localization performance in GNSS-denied scenarios, we develop a dynamic frame interpolation and tracking approach based on 3-D Gaussian rendering. By synthesizing novel viewpoints, this method improves observation matching and strengthens loop closure detection, enabling robust relocalization in large-scale environments. To validate the effectiveness of the proposed framework, we implement a UAV test system and evaluate its performance on both the publicly available TUM dataset and a self-collected aerial dataset spanning several kilometers. Experimental results show that LD3DGS-SLAM achieves a state-of-the-art average localization error of only 0.8 m using just a monocular camera, even during long-range flight missions exceeding tens of kilometers and altitudes up to 500 m. Overall, LD3DGS-SLAM effectively addresses the limitations of monocular SLAM in aerial scenarios, providing a robust, accurate, and cost-effective localization solution for urban air mobility (UAM) and Aerial Internet of Things (IoT) applications.

Simultaneous localization and mapping (SLAM) has attracted tremendous interest from members of the research community in recent years because of its ability to make the robot truly independent in navigation. The integration of neural rendering and the SLAM system showed promising results in joint localization and photorealistic view reconstruction. However, currently available methods that fully rely on implicit representations are resource hungry. Recent work has shown that 3D Gaussians enable high-quality reconstruction and real-time rendering of scenes using multiple posed cameras. Owing to the increasing popularity of Gaussian splatting (GS) and its expanding research area, we present a comprehensive survey of GS based SLAM papers. This survey addresses the challenges of neural based systems by exploring the application of GS within SLAM frameworks. We present a comprehensive analysis of Gaussian Splatting for scene representation in SLAM, focusing on the past three years of research. Our survey delves into the advantages of GS based SLAM, including efficiency, unified representation for various tasks, suitability for resource-constrained devices, and high accuracy and robustness for dense scene reconstruction. Additionally, the survey includes a broader range of research, including semantic GS techniques, to provide a holistic understanding of this approach. By offering an in-depth exploration of Gaussian Splatting in SLAM, a benchmark comparison of performance, and a focus on 3D Gaussian Splatting (3DGS) based SLAM advantages, this survey aims to be a valuable resource for researchers and developers seeking to leverage Gaussian Splatting for efficient and robust SLAM.

Anti-aliasing is a crucial research topic in computer graphics, which can significantly enhance the rendering quality of neural radiance fields (NeRF). Recent studies have introduced effective anti-aliasing NeRF methods, utilizing cone-casting to replace ray-casting and modeling the 3D observation area of pixels as circular cones. The cone-casting strategy has successfully reduced blurring and aliasing in novel view rendering. However, we have observed that the light cones are not standard circular cones because the camera projection model distorts them into elliptic cones of diverse sizes and shapes. This finding motivates us to model pixel light cones as anisotropic elliptic cones and propose an elliptic cone-casting-based anti-aliasing NeRF method called “ECC-NeRF". Specifically, we first derive the elliptic cone models for common pinhole, fisheye, and panoramic cameras based on their camera projection models. Then, we integrate the proposed elliptic cone-casting into two representative cone-casting-based anti-aliasing NeRF methods: Mip-NeRF and Zip-NeRF. Our experimental evaluations on multiple datasets demonstrate that our method can achieve more accurate multi-scale anisotropic representation and better novel view rendering quality with negligible additional computation cost.

Estimating the 3D human poses of multiple individuals using multiple cameras is a significant research topic within the field of vision-based measurement. Contrary to the classical outside-in camera capture system, the inside-out panoramic camera capture system can cover larger scenes with fewer cameras for 3D human pose estimation. This advancement extends the application of 3D human pose measurement beyond small spaces like motion capture studios. For example, this approach can be utilized for intelligent security surveillance in broad outdoor squares or capturing athletes’ movements in large-scale sports scenes. However, existing inside-out 3D human pose estimation methods that utilize panoramic cameras encounter challenges, particularly in multi-person occlusion scenarios. Aimed at these problems, this paper presents a novel inside-out multi-person 3D human pose estimation method using just a few calibrated panoramic cameras. Specifically, we first propose a cross-view multi-person matching algorithm based on panoramic camera epipolar geometry constraint to improve human body matching robustness across viewpoints. Then, we take advantage of multiple panoramic cameras and introduce a multi-view human pose clustering and fusion algorithm to improve the average recall of 3D human pose estimation. In addition, we propose a multi-view human pose nonlinear optimization algorithm to jointly optimize the weighted reprojection errors of estimated 3D human poses, which can further improve the average precision. We have conducted extensive experiments on the public Panoptic Studio dataset and self-built real and simulated datasets to demonstrate that our method can inside-out estimate 3D human poses using multiple panoramic cameras. Compared to state-of-the-art methods, the 3D human pose omission is greatly reduced through the complementarity of multiple cameras, and the precision of 3D human pose estimation is largely improved by utilizing multi-camera observation information.