Bo Wang

Underwater object detection has higher requirements of running speed and deployment efficiency for the detector due to its specific environmental challenges. Nonmaximum suppression (NMS) of two- or one-stage object detectors and transformer architecture of query-based end-to-end object detectors are not conducive to deployment on underwater embedded devices with limited processing power. As for the detrimental effect of underwater color cast noise, recent underwater object detectors make network architecture or training complex, which also hinders their application and deployment on uncrewed underwater vehicles (UUVs). In this article, we propose the underwater detection ConvNet with improved denoising training (U-DECN), the query-based end-to-end object detector (with ConvNet encoder–decoder architecture) for underwater color cast noise that addresses the above problems. We integrate advanced technologies from detection transformer (DETR) variants into detection ConvNet (DECO) and design optimization methods specifically for the ConvNet architecture, including deformable convolution in self-interaction module (SIM) and separate contrastive denoising (CDN) forward methods. To address the underwater color cast noise issue, we propose an underwater color denoising (ColorDN) query method to improve the generalization of the model for the biased object feature information by different color cast noise. Our U-DECN, with ResNet-50 backbone, achieves the best 64.0 AP on DUO and the best 58.1 AP on RUOD, and 21 FPS (five times faster than deformable DETR and DINO 4 FPS) on NVIDIA AGX Orin by TensorRT FP16, outperforming the other state-of-the-art query-based end-to-end object detectors. The code is available at https://github.com/LEFTeyex/U-DECN

The main problem in RGB-T tracking is the correct and optimal merging of the cross-modal features of visible and thermal images. Some previous methods either do not fully exploit the potential of RGB and TIR information for channel and spatial feature fusion or lack a direct interaction between the template and the search area, which limits the model’s ability to fully utilize the original semantic information of both modalities. To address these limitations, we investigate how to achieve a direct fusion of cross-modal channels and spatial features in RGB-T tracking and propose CSTNet. It uses the Vision Transformer (ViT) as the backbone and adds a Joint Spatial and Channel Fusion Module (JSCFM) and Spatial Fusion Module (SFM) integrated between the transformer blocks to facilitate cross-modal feature interaction. The JSCFM module achieves joint modeling of channel and multi-level spatial features. The SFM module includes a cross-attention-like architecture for cross modeling and joint learning of RGB and TIR features. Comprehensive experiments show that CSTNet achieves state-of-the-art performance. To enhance practicality, we retrain the model without JSCFM and SFM modules and use CSNet as the pretraining weight, and propose CSTNet-small, which achieves 50% speedup with an average decrease of 1-2% in SR and PR performance. CSTNet and CSTNet-small achieve real-time speeds of 21 fps and 33 fps on the Nvidia Jetson Xavier, meeting actual deployment requirements. Code is available at https://github.com/LiYunfengLYF/CSTNet.

Simultaneous localization and mapping (SLAM) based on point features is widely used in precision missions for unmanned underwater vehicles (UUVs). However, the inherent complexity of underwater environments poses significant challenges for point feature-based SLAM, such as reduced pose estimation accuracy due to factors like turbidity and weak texture, as well as limited semantic information conveyed by sparse point cloud maps. To address these challenges, this study introduces an innovative approach termed underwater semantic object SLAM (USO-SLAM), based on stereo camera. We propose a novel algorithm designed for reconstructing objects that naturally occur underwater, thereby substantially enhancing reconstruction success rate and object pose estimation accuracy. Furthermore, we present a comprehensive underwater semantic object SLAM algorithmic framework, incorporating a robust semantic loop closure module whereby objects are encoded within a two-layers topological graph structure. Empirical findings derived from both simulated and real-world underwater environments corroborate that our proposed method outperforms existing state-of-the-art semantic reconstruction algorithms and SLAM systems, particularly in terms of semantic object reconstruction and navigation accuracy in underwater settings.