Cloud Image Retrieval for Sea Fog Recognition (CIR-SFR) Using Double Branch Residual Neural Network
Open Access
- 14 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 16 (19391404) , 3174-3186
Abstract
Sea fog is a common weather phenomenon at sea, which reduces visibility and causes tremendous hazards to marine transportation, marine fishing and other maritime operations. Traditional sea fog monitoring methods have enormous difficulties in characterizing the diversity of sea fog and distinguishing sea fog from low-level clouds. Thus, we propose a cloud image retrieval method for sea fog recognition (CIR-SFR) in a deep learning framework by combining the advantages of metric learning. CIR-SFR includes the feature extraction module and the retrieval-based sea fog recognition module. The feature extraction module adopts the Double Branch Residual Neural Network (DBRNN) to comprehensively extract the global and local features of cloud images. By introducing local branches and using activation masks, DBRNN can focus on regions of interest in cloud images. Moreover, cloud image features are projected into the semantic space by introducing Multi-Similarity Loss, which effectively improves the discrimination ability of sea fog and low-level clouds. For the retrieval-based sea fog recognition module, similar cloud images are retrieved from the cloud image dataset according to the distance in the feature space, and accurate sea fog recognition results are obtained by counting the percentage of various cloud image types in the retrieval results. To evaluate the sea fog recognition system, we establish a dataset of 2544 cloud images including clear sky, low-level cloud, medium high cloud and sea fog. Experimental results show that the proposed method outperforms the traditional methods in sea fog recognition, which provides a new way for sea fog recognition.Keywords
Funding Information
- National Natural Science Foundation of China (42071323)
- Public Welfare Science and Technology Project of Ningbo (202002N3104)
This publication has 25 references indexed in Scilit:
- Deep Metric Learning for Visual Understanding: An Overview of Recent AdvancesIEEE Signal Processing Magazine, 2017
- Development of a daytime cloud and haze detection algorithm for Himawari‐8 satellite measurements over central and eastern ChinaJournal of Geophysical Research: Atmospheres, 2017
- Deep Residual Learning for Image Recognition2016
- Deep Supervised Hashing for Fast Image Retrieval2016
- Deep learning of binary hash codes for fast image retrieval2015
- Satellite cloud image processing and information retrieval system2012
- Optical Radiance Characteristics of Sea Fog Based on Remote SensingActa Optica Sinica, 2008
- Dimensionality Reduction by Learning an Invariant Mapping2006
- Query-by-shape in meteorological image archives using the point diffusion techniqueIEEE Transactions on Geoscience and Remote Sensing, 2001
- Beiträge zur Optik trüber Medien, speziell kolloidaler MetallösungenAnnalen der Physik, 1908