Renke Kou

The use of local contrast and edge information as prior knowledge is common in hybrid InfraRed Small Target Detection (IRSTD) methods. However, existing methods tightly integrate prior knowledge modules with the backbone network, limiting their flexibility. To address this issue, we propose a Lightweight Local Contrast and Edge Fusion Approach (LLCEFA), featuring a dual-branch architecture with a main branch and an edge branch designed to fully leverage the synergistic advantages of prior knowledge guidance and data-driven learning. Within the main branch, the proposed Local Contrast Attention Fusion (LCAF) module leverages the local contrast prior to reconstruct the input image into more discriminative feature representations. Concurrently, the edge branch employs the developed Adaptive Edge Weight Extraction (AEWE) module to independently derive edge weights for enhancing intermediate-layer features. This design effectively decouples prior knowledge processing from the backbone's architecture, allowing LLCEFA to serve as a plug-and-play enhancer compatible with various backbone methods, achieving a performance boost with introducing only a minimal additional burden. Experimental results validate our LLCEFA's efficacy in enhancing backbone networks, achieving average performance gains of 2.233%,1.826%, and 1.056% in Intersection over Union ($IoU$), normalized Intersection over Union ($nIoU$), and Probability of Detection (${P_{d}}$), respectively, while False-alarm Rate (${F_{a}}$) is reduced by an average of 6.729$\times 10{^{-6}}$ across three public datasets. This comes at an average increase of only 0.0009M parameters and 0.0617G Floating-Point Operations (FLOPs) added to the backbone, highlighting its high efficiency and broad applicability.

The data-driven InfraRed Small Target Detection (IRSTD) methods have witnessed remarkable advancements in performance. However, these methods typically rely on high-quality pixel-level mask labels, requiring substantial human effort and time for annotation. To tackle this challenge, we propose a Prior Knowledge Enhanced Learning Approach (PKELA), which contains two key strategies: the Prior Knowledge Enhanced Initial Pseudo-label Generation (PKEIPG) strategy and the Teacher Knowledge Guided Label Update (TKGLU) strategy. Specifically, for the PKEIPG strategy, we introduce a Local Contrast Enhancement (LCE) module based on prior knowledge to suppress background interference. Drawing inspiration from the human brain’s processing sequence of visual information, we locate an appropriate neighborhood containing the infrared small target by identifying the target’s edges. Within this neighborhood, high-quality initial pseudo labels are obtained through adaptive threshold segmentation, which provide strong supervisory signals for the data-driven IRSTD method during the initial training phase. For the TKGLU strategy, we employ an Exponential Moving Average (EMA) teacher model that dynamically updates its parameters based on the current model’s state. Furthermore, a memory bank is established to archive the teacher model’s performance, which is treated as prior knowledge. This knowledge is then systematically used to guide the refinement of pseudo labels. Experimental results on the SIRST, NUDT-SIRST, IRSTD-1k, and SIRST3 datasets indicate that our PKELA exhibits superior training performance with single-point labels. By incorporating our approach, the data-driven IRSTD methods could achieve near-full-supervised performance in terms of Probability of Detection (Pd) metric, with Intersection over Union (IoU) and normalized Intersection over Union (nIoU) reaching up to 95.75% and 95.82% of full-supervised performance, respectively. Our code is available at https://gitee.com/mynewspace/pkela.

Infrared small target detection plays a crucial role in infrared search and tracking systems. However, current detection methods are limited by the small target size and low signal-to-noise ratio of infrared imagery. Furthermore, motion features for target detection are difficult to extract using simple frame subtraction due to poor imaging conditions. Therefore, we focus on the holistic interframe interaction to enhance the temporal feature and propose a spatiotemporal local contrast method in this letter. First, the motion-enhanced density peak clustering is employed to determine the robust localization of candidate targets, in which the density feature maps are generated by the preprocessing of non-consecutive three frames difference after image registration. Second, to reliably exploit interframe interactions across both non-consecutive and successive frames, a temporal domain saliency map is computed based on local regions from successive frames. Moreover, a spatial domain saliency map is obtained using a novel tri-layer local contrast measure. By fusing results from both domains, the infrared small targets are detected through adaptive threshold segmentation. Experimental results on four real sequences demonstrate that the proposed method can achieve better detection performance by target enhancement and background suppression than other spatiotemporal algorithms.

We pioneer a learning-based single-point prompt paradigm for infrared small target label generation (IRSTLG) to lobber annotation burdens. Unlike previous clustering-based methods, our intuition is that point-guided mask generation just requires one more prompt than target detection, i.e., IRSTLG can be treated as an infrared small target detection (IRSTD) with the location hint. Therefore, we propose a simple yet effective Energy-Double-Guided Single-point Prompt (EDGSP) framework, aiming to adeptly transform a coarse IRSTD network into a refined label generation method. Specifically, EDGSP comprises three key modules: 1) target energy initialization (TEI), which establishes a foundational outline to streamline the mapping process for effective shape evolution, 2) double prompt embedding (DPE) for rapidly localizing interesting regions and reinforcing high-resolution individual edges to avoid label adhesion, and 3) bounding box-based matching (BBM) for eliminating false masks via considering comprehensive cluster boundary conditions to obtain a reliable output. In this way, pseudo labels generated by three backbones equipped with our EDGSP achieve 100% objectlevel probability of detection (Pd) and 0% false-alarm rate (Fa) on SIRST, NUDT-SIRST, and IRSTD-1k datasets, with a pixel-level intersection over union (IoU) improvement of 13.28% over stateof-the-art (SOTA) label generation methods. Further applying our inferred masks to train detection models, EDGSP, for the first time, enables a single-point-generated pseudo mask to surpass the manual labels. Even with coarse single-point annotations, it still achieves 99.5% performance of full labeling. Code is available at https://github.com/xdFai/EDGSP.

Identifying challenging samples in real-time aerial images captured by UAVs is currently a difficult task. Lightweight network designs often utilize lightweight modules or employ methods such as network pruning, quantization, and distillation for real-time detection on embedded devices. However, these approaches are inadequate for complex background aerial images. In this paper, we propose a real-time object detection algorithm specifically designed for complex aerial images, named MINIAOD. First, we use Ghost convolution to build a lightweight backbone network, which improves the detection speed without compromising the feature extraction capability. Additionally, we utilize the GSConv module to construct a feature enhancement network and develop the GSC3ECA module to improve the network’s learning capability for challenging targets, while also decreasing computational complexity. To boost the network’s focus on small and medium-sized targets and mitigate interference from complex backgrounds, we incorporate the ECA attention mechanism in the small and medium-sized target detection branch to elevate the focus on difficult-to-detect samples. Finally, we design the rotation detection head to characterize the target directionality of aerial images. We perform extensive experiments on multiple publicly available datasets and the proposed method demonstrates a favorable balance between accuracy and speed when compared to the current leading models.

The sparse characteristics of target features poses significant challenges when using deep learning methods for infrared dim small targets. To tackle this issue, this article proposes a novel multilevel sparse feature fusion network for detecting infrared dim small targets. A feature-level sparse feature fusion network fuses target features of the same level and different depths to express small target features. A decision-level sparse feature fusion network fuses features from different decision spaces to improve decision confidence. To enrich the feature representation of the target, different levels of target global features are introduced into the decision-level sparse feature fusion network. During the network training process, a deep joint supervision training strategy is proposed to supervise and train the multilevel sparse feature fusion network, aiming to fully learn the feature representation of the target. According to the experimental results, the proposed infrared dim small targets detection method outperforms existing popular methods under sparse target features.

Accurate building semantic segmentation in remote sensing imagery is essential for urban planning, environmental monitoring, and map creation. While deep learning has achieved significant advancements in this field, precisely segmenting building edges and shadows in complex scenarios remains challenging. Shadows often introduce boundary ambiguities, affecting the local shape and texture information of buildings. Current methods do not fully perceive or utilize shadows. To address these challenges, we propose an advanced high-resolution image segmentation network, high-resolution network, integrated with a shadow-inclusive edge perception module. Our approach involves introducing a shadow-inclusive contour transition module (SCTM) during the feature extraction stage to enhance the features of blurry boundaries. The proposed SCTM and shadow-aware attention module significantly enhance attention maps, improve responses in blurry boundary regions, and increase consistency between predictions and ground truth, setting a new benchmark for building semantic segmentation in remote sensing imagery. This enriched information is then fed into an attention module that concurrently focuses on boundary and channel features, surpassing traditional semantic segmentation methods. We validated our method on three datasets: Massachusetts, WHU, and Inria. Our approach outperformed state-of-the-art methods on the WHU Building Dataset across all metrics, including mIoU, Accuracy, Kappa, and Dice coefficient.

With the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images (ARSI) fail to achieve around-the-clock object detection. To tackle this, we propose an around-the-clock object detection algorithm YOLOFIV that fuses infrared and visible features. First, we design a dual-stream backbone network based on the attention mechanism to adequately extract the features of both modalities. Moreover, the ECA attention mechanism is integrated into the feature enhancement network to amplify attention toward challenging detection scenarios. Finally, we improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method YOLOFIV on the widely used Drone Vehicle dataset, YOLOFIV achieves an accuracy of 64.71% (in terms of $mAP_{0.5}$ ), accuracy improvement of 8.32% over baseline bimodal model, similar performance to UACMD designed for ARSI object detection but with 92.35% reduction in parameter count, and 17.87 times speedup. The results show that our approach achieves round-the-clock object detection while maintaining a favorable accuracy-speed trade-off.

Due to the lack of color and texture information and the fuzzy boundary of infrared (IR) small targets, the pixel-level mask annotation process consumes a lot of manual costs and makes it difficult to achieve accurate annotation. To further reduce the annotation burden, we propose an IR small target mask generation algorithm based on single-point supervised multiscale chain growth clustering (MCGC). The core of this work is the adaptive generation of IR small target pseudo mask maps under the supervision of randomly given single-point labels, sequentially through the strategies of multiscale chain growth, Euclidean coefficient attenuation, K-Means clustering, and eight-neighborhood clustering. On the four public datasets, ablation experiments, and qualitative and quantitative comparison experiments demonstrate that the MCGC algorithm has an efficient and accurate IR small target pseudo mask generation capability, which can be adapted to different numbers, scales, shapes, and intensities of targets in complex backgrounds. In addition, IR-Labelmask, an IR small target mask annotation software designed based on the MCGC algorithm, is publicly available on kourenke/IR-Labelmask-software (github.com). To our knowledge, this is the first mask annotation software designed for IR small targets.