Instance-Aware Spatial-Frequency Feature Fusion Detector for Oriented Object Detection in Remote-Sensing Images
- 6 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 61 (01962892) , 1-13
Abstract
In recent years, fusing multi-type features poses great potential for oriented object detection (OOD) in remote sensing images (RSIs). Due to the inexplicit operation of modeling orientation variations, convolutional neural networks (CNNs) are difficult to perceive objects under different transformations (angles and scales). In this paper, we propose a novel instance-aware spatial-frequency feature fusion detector (SFFD) for oriented object detection in remote sensing images. First, a layer-wise frequency-domain analysis (L-FDA) module is built along with CNN layers to extract frequency features. Getting rid of the constrains such as horizontal rectangular kernel in CNNs, our L-FDA possesses outstanding ability of locating mutational signals from frequency space. These mutational signals record the scale and angle information of the oriented instances in images. Subsequently, CNN and frequency features are sent into RoI Pooling layer to obtain multi-type instance-level RoI features. Moreover, the proposed instance-aware cross feature fusion (CFF) module explores the interaction between these diverse features which provides an explicit indicator to compensate the orientation information ignored by instance-level CNN features. Finally, our SFFD unifies the proposed L-FDA module and CFF module into the detection network to localize oriented instances in RSIs. We compare our method with many state-of-the-art methods on DOTA, HRSC2016, and NWPU VHR-10 datasets. Experimental results verify the validity of modeling instance-level object relations from frequency-domain and CNNs for OOD.Keywords
Funding Information
- National Natural Science Foundation of China (61971223, 61976117, 62071233)
- Key Projects of University Natural Science Fund of Jiangsu Province (19KJA360001)
- Jiangsu Provincial Natural Science Foundation of China (BK20180018, BK20191409, BK20211570)
- Fundamental Research Funds for the Central Universities (30917015104, 30919011103, 30919011402, 30921011209, JSGP202204)
This publication has 58 references indexed in Scilit:
- Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing ImagesIEEE Transactions on Geoscience and Remote Sensing, 2016
- Deep Residual Learning for Image Recognition2016
- Microsoft COCO: Common Objects in Context2014
- Analysis of the Proportion of Surface Reflected Radiance in Mid-Infrared Absorption BandsIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013
- Rotation-Invariant HOG Descriptors Using Fourier Analysis in Polar and Spherical CoordinatesInternational Journal of Computer Vision, 2013
- A Comparative Study on Linear Regression-Based Noise Estimation for Hyperspectral ImageryIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013
- Joint DOA and multi-pitch estimation based on subspace techniquesEURASIP Journal on Advances in Signal Processing, 2012
- Adaptive Markov Random Field Approach for Classification of Hyperspectral ImageryIEEE Geoscience and Remote Sensing Letters, 2011
- Endmember Extraction of Hyperspectral Remote Sensing Images Based on the Ant Colony Optimization (ACO) AlgorithmIEEE Transactions on Geoscience and Remote Sensing, 2011
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009