Huihui Li

Synthetic Aperture Radar (SAR) is widely used in military and civilian applications due to its all-weather, day-and-night imaging capability. However, interpreting SAR images is challenging for both experts and non-experts. Inspired by deep learning-based style transfer, researchers have employed Generative Adversarial Networks (GANs) to convert SAR images into more intuitive optical ones. Yet, current loss functions and evaluation metrics focus mainly on pixel-level differences, overlooking the structural coherence required for target recognition and downstream tasks. To address this, we propose a semantic segmentation-driven framework for target-level SAR-to-optical image translation. Compatible with various supervised models, it incorporates segmentation loss and uses SAR segmentation maps as additional inputs to preserve target structure. Experiments on custom datasets, built from Sentinel 1-2 imagery with road binary segmentation labels, as well as public datasets, confirm the method's effectiveness across different base translation models. The source code and the datasets used will be published at the following URL https://github.com/NWPU-LHH/SOIT-Seg-Driven.

Synthetic Aperture Radar (SAR), an active remote sensing technology that can be deployed on various aerial plat forms, generates images by emitting microwaves and analyzing the intensity of backscattered signals. The penetration capability of microwaves enables observation of Earth at all weather, day and night, broadening the application of SAR. However, SAR image interpretation remains challenging for non-experts. In contrast, optical images provide intuitive visual features but are weather-sensitive. To synergize these modalities, SAR-to-Optical Translation (S2OT) has gained research attention, but most studies prioritize visual quality or similarity metrics over practical applicability to downstream tasks. This study proposes a high level vision task-coordinated S2OT framework to address this gap. Semantic segmentation, emulating land cover classification, is integrated as the downstream task. Semantic segmentation loss guides the network to generate optical images that enhance task-relevant features. To preserve structural information in SAR images, we introduce SSIM loss and incorporate SAR derived semantic segmentation maps as auxiliary inputs. An identity loss further aligns the distributions of generated and real optical images, mitigating domain discrepancies. Extensive experiments confirm that S2OT improves downstream land cover classification. The inclusion of task-specific losses elevates translation quality: our HVTC-GAN surpasses the baseline methods in SSIM and PSNR metrics. Ablation studies validate the effectiveness of co-training S2OT with high-level vision tasks, demonstrating that task-oriented constraints enhance both translation fidelity and downstream utility. Code will be available at https://github.com/NWPU-LHH/HVTC-GAN

Optical remote sensing and Synthetic Aperture Radar(SAR) remote sensing are crucial for earth observation, offering complementary capabilities. While optical sensors provide high-quality images, they are limited by weather and lighting conditions. In contrast, SAR sensors can operate effectively under adverse conditions. This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN, designed to enhance the accuracy of ship target translation by leveraging semantic information from a pre-trained semantic segmentation model. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network, improving the quality of output optical-styled images. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed. This work suggests broader research and applications for downstream-task-guided frameworks. The code and link to download proposed HRSID-DIOR dataset will be available at https://github.com/NPULHH/.

Image-text retrieval has made remarkable achievements through the development of feature extraction networks and model architectures. However, almost all region featurebased methods face two serious problems when modeling modality interactions. First, region features are prone to feature entanglement in the feature extraction stage, making it difficult to accurately reason complex intra-model relations between visual objects. Second, region features lack rich contextual information, background, and object details, making it difficult to achieve precise inter-modal alignment with textual information. In this paper, we propose a novel Dual Stream Relation Learning Network (DSRLN) to jointly solve these issues with two key components: a Geometry-sensitive Interactive Self-Attention (GISA) module and a Dual Information Fusion (DIF) module. Specifically, GISA extends the vanilla self-attention network from two aspects to better model the intrinsic relationships between different regions, thereby improving high-level visual-semantic reasoning ability. DIF uses grid features as an additional visual information source, and achieves deeper and complex fusion between the two types of features through a masked crossattention module and an adaptive gate fusion module, which can capture comprehensive visual information to learn more precise inter-modal alignment. Besides, our method also learns a more comprehensive hierarchical correspondence between images and sentences through local and global alignment. Experimental results on two public datasets, i.e., Flickr30K and MS-COCO, fully demonstrate the superiority and effectiveness of our model.

Pointer instruments are widely used in various fields due to their simplicity, low cost, strong anti-interference capabilities, and durability. However, manual reading and recognition of these instruments can be complex and labor-intensive. Existing meter recognition algorithms have limitations in dial extraction and reading calculation, which restrict their practical applications. In this paper, we propose an instrument detection method based on arbitrary quadrilateral positioning. It directly locates key areas such as pointers and scales, eliminating irrelevant information and simplifying the reading process. By applying perspective transformation to the detected quadrilateral, the instrument is transformed into a front-view state, eliminating the need for time-consuming post-processing operations. We also propose a reading algorithm that handles missing scales, improving reading accuracy. Experimental results demonstrate that our method outperforms existing algorithms in terms of detection and reading accuracy. Using a self-built instrument data set, the mean average precision (mAP) is 92.4, and the absolute error of the readings is less than 0.02. It is particularly effective in challenging scenarios involving tilt, rotation, changing lighting, and small targets, making it suitable for real-world applications.

Recently, remote sensing image-text retrieval (RSITR) has received significant attention due to its flexible query form and effective management of remote sensing images. However, prior work often relies on compact global features and ignores local features that can reflect salient objects in the images. Moreover, these methods primarily model interactions between features in the spatial domain, which is insufficient for mining the rich semantic information presented in remote sensing images. In this article, we propose a novel spatial-channel attention transformer (SCAT) with pseudo regions to address these issues. Concretely, in order to acquire the fine-grained perception of local objects, we introduce a pseudo region generation (PRG) module that adaptively aggregates grid features with similar semantic information into multiple clusters through a clustering algorithm. These generated cluster centers are able to flexibly and efficiently represent local objects in remote sensing images without relying on sophisticated object detectors. Furthermore, in order to achieve a comprehensive understanding of image semantics information, we carefully construct a novel SCAT. By exploiting spatial and channel attention to explore the dependencies between features at both spatial and channel domains, the proposed SCAT enhances the model’s ability to identify both “where to look” and “what it is,” thereby obtaining a more powerful representation. In addition, SCAT incorporates two novel designs that alleviate the high overhead caused by attention modeling. Extensive experiments on two benchmark datasets, RSICD and RSITMD, fully demonstrate the effectiveness and superiority of our proposed method.

Current image-text retrieval methods mainly utilize region features that provide object-level information to represent images, making the retrieval results more accurate and interpretable. However, there are several issues with region features, such as lack of rich contextual information, loss of object details and risk of detection redundancy. The ideal visual features in image-text retrieval should have three characteristics: object-level, semantically-rich, and language-aligned. To this end, we propose a novel visual representation framework to capture more comprehensive and powerful visual features. Specifically, since these region feature disadvantages are the grid feature advantages, we first build a two-step interaction model to explore the complex relationship between them from the spatial and semantic perspectives to integrate their complementary information, making the fused visual features both object-level and semantic-rich. Then, we design a text-integrated visual embedding module that utilizes textual information as guidance to filter redundant regions, further endowing visual features with language-aligned capabilities. Finally, we develop a multi-attention pooling module to better aggregate these enhanced visual features in a more fine-grained manner. Extensive experiments demonstrate that our proposed model achieves state-of-the-art performance on the benchmark datasets Flickr30K and MS-COCO.

The synthetic aperture radar (SAR) system is a kind of active remote sensing, which can be carried on a variety of flight platforms and can observe the Earth under all-day and all-weather conditions, so it has a wide range of applications. However, the interpretation of SAR images is quite challenging and not suitable for nonexperts. In order to enhance the visual effect of SAR images, this article proposes a multiscale generative adversarial network based on wavelet feature learning (WFLM-GAN) to implement the translation from SAR images to optical images; the translated images not only retain the key content of SAR images but also have the style of optical images. The main advantages of this method over the previous SAR-to-optical image translation (S2OIT) methods are given as follows. First, the generator does not learn the mapping from SAR images to optical images directly but learns the mapping from SAR images to wavelet features and then reconstructs the gray-scale images to optimize the content, increasing the mapping relationships and helping to learn more effective features. Second, a multiscale coloring network based on detail learning and style learning is designed to further translate the gray-scale images into optical images, which makes the generated images have an excellent visual effect with details closer to real images. Extensive experiments on SAR image datasets in different regions and seasons demonstrate the superior performance of WFLM-GAN over the baseline algorithms in terms of structural similarity (SSIM), the peak signal-to-noise ratio (PSNR), the Frechet inception distance (FID), and the kernel inception distance (KID). Comprehensive ablation studies are also carried out to isolate the validity of each proposed component. Our codes will be available at https://github.com/G2022G/WFLM-GAN .