Remote Sensing Teacher: Cross-Domain Detection Transformer With Learnable Frequency-Enhanced Feature Alignment in Remote Sensing Imagery
- 18 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-14
Abstract
Unsupervised Domain Adaptation (UDA) is critical for remote sensing object detection in real applications, aiming to address the significant performance degradation issue caused by the domain gap between source and target domain. This method achieves cross-domain alignment by leveraging the unlabeled target domain data, thus avoiding the expensive annotation cost. However, existing works mainly cope with CNN-based object detectors, which are characterized by complex adversarial learning architecture and fail to accurately align the features in remote sensing images with sparsely allocated objects and inevitable background noise. Compared to CNN-based methods, the DEtection TRansformer (DETR) largely simplifies the object detection pipeline and demonstrates the great potential by its intrinsic characteristics of global relation modeling between any pixels. On this basis, we propose the first strong DETR-based baseline, Remote Sensing Teacher, for unsupervised domain adaptation in remote sensing object detection. Specifically, the Remote Sensing Teacher introduces an innovative Learnable Frequency-enhanced feature Alignment (LFA) module. Within this module, we initially transform the features into frequency space to simplify the attention solver and effectively capture domain-specific information. Subsequently, the module significantly enhances the global feature representations of sparsely allocated objects by using a lightweight attention mechanism. Following this, the module incorporates learnable filters with a gated mechanism, enabling selective alignment of features in noisy backgrounds. Additionally, the Remote Sensing Teacher employs a Self-adaptive Pseudo label Assigner (SPA) that can automatically adjust the class-wise confidence threshold according to the model’s learning status, thereby enabling the generation of high-quality pseudo-labels in scenarios with a long-tailed distribution. Leveraging these pseudo-labels further mitigates the domain bias of the detector by establishing alignment at the label level. Extensive experimental results demonstrate superior performance and generalization capabilities of our proposed Remote Sensing Teacher in multiple remote sensing adaptation scenarios. Code is released at https://github.com/h751410234/RemoteSensingTeacher.Keywords
Funding Information
- MYHT Program (D040404)
- National Natural Science Foundation of China (62301046)
- National Key Laboratory for Space-Born Intelligent Information Processing (TJ-01-22-01)
This publication has 43 references indexed in Scilit:
- SSDA-YOLO: Semi-supervised domain adaptive YOLO for cross-domain object detectionComputer Vision and Image Understanding, 2023
- MTTrans: Cross-domain Object Detection with Mean Teacher Transformer2022
- Every Pixel Matters: Center-Aware Feature Alignment for Domain Adaptive Object Detector2020
- End-to-End Object Detection with Transformers2020
- Domain Adaptive Faster R-CNN for Object Detection in the Wild2018
- Ship detection in SAR images based on an improved faster R-CNN2017
- Deep Residual Learning for Image Recognition2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- ImageNet: A large-scale hierarchical image database2009
- EMA: A process model of appraisal dynamicsCognitive Systems Research, 2009