Ju He

Side-scan sonar (SSS) imaging plays a critical role in underwater perception for autonomous underwater vehicles (AUVs). However, the spatial sparsity of targets and the limited computational resources remain challenging for real-time object detection. Existing methods typically adopt dense inference strategies, leading to substantial computational redundancy and limited deployment feasibility. In this work, we propose a lightweight and ultra-fast SSS object detection framework based on target presence awareness. The proposed framework follows a coarse-to-fine inference paradigm, in which a target presence analysis module is first employed to rapidly filter out target-absent image patches, and only target-positive patches are forwarded to an Object Forward Detection (OFD) module for fine-grained detection. The TPA module integrates spatial–frequency convolution to efficiently capture both local structural cues and global contextual information with minimal computational overhead. Furthermore, an AttnConv-enhanced detection module is introduced in the OFD stage to strengthen high-frequency target features and improve fine-grained detection performance. Extensive experiments on public SSS datasets demonstrate that the proposed method achieves an mAP of 74.63% on the AI4Shipwrecks dataset and 63.02% on the SSS-Mine dataset. Notably, the framework delivers an ultra-fast inference speed of 174.74 FPS on embedded hardware, representing a 5.2× speedup over conventional dense-processing detection methods.

Forward-looking sonar (FLS) image segmentation enables underwater robotics autonomous exploration in marine engineering. However, the inherent characteristics of sonar systems such as weak semantic information, complex backgrounds, and high environmental noise, pose significant challenges and cause performance bottlenecks for traditional FLS image segmentation algorithms. Moreover, relying on single-modal features further exacerbates these limitations, restricting the segmentation accuracy and robustness. To address these limitations, this paper proposes FLSFusionNet which models the multi-modal relationships between sonar space and vision modality, capturing complementary features and enhancing segmentation performance through adaptive cross-modal interactions. Specifically, FLSFusionNet comprises four key modules: (1) sonar and camera backbones that extract distinct features from each modality; (2) a camera-sonar spatial transformer module designed to bridge the representation gap between sonar and visual features; (3) a cross-modal adaptive fusion module that maintain the semantic consistency between the dual features for effective integration; and (4) a fusion-based segmentation head that predicts the final segmentation results. Experiments conducted on the multi-modal benchmark dataset demonstrate that the proposed fusion approach significantly outperforms existing single-modal and multi-modal sonar image methods in segmentation accuracy, especially under complex underwater conditions. The results underscore the potential of sonar-vision fusion strategy to enhance the robustness and precision of FLS-based applications.

In recent years, autonomous docking technology for autonomous surface vessels (ASVs) has attracted increasing research interest, and boat berth detection plays a crucial role in this task. However, common berth detection methods for ASVs adopt specific marks, such as ArUco codes, to implement boat berth detection, which significantly limits their applicability in practical deployments, since most conventional wharfs do not contain such markers. To support tasks involving human-less return to dock and enhance the generalizability of berth detection in real-world wharf scenarios, this article proposes a novel markless berth detection framework based on high-level scene semantics, without relying on artificial markers. Our proposed detection framework, based on the fusion of camera and light detection and ranging (LiDAR), comprises two important components. First, a LiDAR-camera fusion point cloud semantic segmentation model is constructed to generate 3-D semantic point clouds. Second, a berth extraction algorithm is designed to process point cloud segmentation results and detect free berths for ASVs. In addition, to improve the performance of the point cloud semantic segmentation model for ASV scenes, we developed a LiDAR-assisted view transformer module and a LiDAR-camera fusion attention module. Moreover, in our collected point cloud segmentation data set for ASV wharf scenes, our proposed method outperforms other point cloud segmentation baselines. Moreover, we conducted real-world prototype experiments to validate the feasibility and robustness of our markless boat docking detection method. The experimental results demonstrate that our proposed berth detection method has a high potential to support applications involving the autonomous docking of ASVs.

Marine object detection is an essential component for maritime autonomous surface ship (MASS) systems, enabling them to perceive obstacle objects and avoid them. Existing detection algorithms primarily rely on visual sensors and face serious limitations in complex maritime environments. In this paper, we propose a novel joint object detection method based on the fusion of marine radar and camera data for robust maritime object detection. By employing a multi-stage hybrid-level fusion architecture, the proposed method efficiently fuses marine radar data with camera images, thereby enhancing the accuracy and robustness of object detection in complex maritime contexts. In addition, inspired by vision transformer detectors, a new transformer-based fusion module is proposed to mitigate semantic disparity between the two modalities and enhance radar information guidance through extracted object queries. Extensive experiments on public maritime perception datasets demonstrate the significant advantages of the proposed method in various maritime environments. The framework also exhibits strong generalization capability, making it applicable to diverse maritime settings, including challenging conditions like low visibility and high background noise. In public maritime perception datasets, the proposed method achieves state-of-the-art accuracy compared to other single-modality baselines and radar-camera fusion baselines, outperforming the excellent visual object detection model YOLOv8-x in terms of mAP0.35 increased by 13.72% in common scenarios and 24.85% in complex scenarios.

Forward-looking sonar imaging systems can provide long-range underwater environments in ultra-low visibility and contrast, holding a critical role in the recognition and segmentation of underwater objects. However, sonar image segmentation presents significant challenges due to the main factors: 1) the reverberation strength increased by the coherent scattering phenomenon, which hampers the segmentor’s ability to capture object features accurately, and 2) insufficient multifeature utilization of weakly annotated samples and a lack of prior information on underwater object scales. To overcome this challenge, we proposed a novel forward-looking sonar image segmentation method based on the reverberation suppression and multi-layer context aware. Detailed statistical analysis of reverberation distribution using nonlinear least squares fitting, we design a self-supervised background suppression auxiliary branch to reduce inherent gamma noise and preserve pure features. To further leverage the multi-scale context features extracted from different transformer encoding layers, we propose a multi-layer aggregation transformer as an encoder to acquire more comprehensive target features. Additionally, a global-local feature complementary fusion mechanism was designed for the interaction of these independent features, to achieve an optimal comprehensive state at the spatial and channel level. The comparative experiments demonstrate that FISNet achieves superior performance with state-of-the-art segmentation networks.

Recently, unmanned surface vessels (USVs) have played an increasingly important role in autonomous exploration, and boat detection is an important task for USVs. While most existing boat detection methods focus on 2-dimensional (2D) detection, 3-dimensional (3D) detection which provides valuable spatial direction for moving target estimation has not been studied in the boat detection field. However, 3D boat detection on water surfaces faces challenging problems, such as small sizes of detected targets and diverse moving directions. Considering that traditional LiDAR-based 3D boat detection methods require high hardware costs, we fuse millimeter wave (MMW) radar and high semantic camera to achieve low-cost and high-quality 3D boat detection. We propose a novel radar-camera fusion boat 3D detection model named RCBDet. The proposed RCBDet employs a new dual radar encoder and first introduces Doppler speed information from MMW radar into neural network to overcome sparse radar points. A new radar-camera attention module is designed to effectively combine camera features, radar spatial features and radar velocity features, encapsulating not only shape and semantic attributes but also spatial orientation information. In our collected boat 3D detection dataset, RCBDet achieves state-of-the-art accuracy compared to other single-modality baselines and radar-camera fusion baselines. Moreover, we conducted comprehensive ablation experiments to validate the efficacy of the designed modules. The experimental results demonstrated that the proposed radar-camera fusion model effectively fuses MMW radar features and camera features.