Hui Lin

Remote sensing image change captioning (RSICC) aims to generate sentence descriptions about land cover changes in bitemporal images. The effective acquisition of semantic-level change information is critical for this task. However, due to the effects of illumination interference, appearance similarities and scale differences between different objects, it is difficult to accurately extract change information from bitemporal images. In this article, we attempt to take advantage of the high-level semantic information inherent in text and propose a text-augmented semantic feature extraction and difference information learning model for RSICC. Specifically, we first predefine some text prompts for each remote sensing image and use the contrastive language-image pretraining (CLIP) model to select the most suitable text descriptions for them. Then, we adopt a refined segment anything model (SAM) to learn fine-grained visual features from each image, which is further enhanced via a designed selective text–image fusion (STIF) module. After that, to extract the semantic differences between bitemporal images, we propose a text-guided difference capture (TGDC) module capable of extracting multiscale difference information under the guidance of text differences between different-time images. Finally, a transformer-based caption generator is applied to generate sentence descriptions from the extracted difference information. In order to test the performance of our proposed model, we conduct comprehensive experiments on two widely used RSICC datasets, including LEVIR-CC and Dubai-CC. The experimental results show that our proposed model is able to outperform several state-of-the-art models, which validates the effectiveness of it. The codes of our proposed model will be released at https://github.com/Richardkimyo/TACC

Change detection (CD) is one of the most important methods for monitoring land surface changes. Recently, transformer-based models have been employed to CD. However, most of them focus on modeling the correlation within each image, while cannot well model the difference between bi-temporal images. In this paper, we propose a difference transformer network (DiFormer) to address this issue. Specifically, we propose a Token Exchange-based Difference Evaluation (TEDE) module to generate the inconsistency between the changed region and the surrounding context to highlight the difference between the bi-temporal images. In addition, to obtain semantically rich exchangeable tokens, we design a Multi-scale Semantic Perception (MSP) module, which provides assistance for difference modeling. In order to test the performance of DiFormer, we conduct qualitative and quantitative experiments on two public datasets, including LEVIR-CD and S2Looking. Experimental results show that our proposed DiFormer is able to achieve better results than several state-of-the-art models, with F1 of 92.15% on the LEVIR-CD dataset and 66.31% on the S2Looking dataset.