Adaptive Self-Supporting Prototype Learning for Remote Sensing Few-Shot Semantic Segmentation
- 29 July 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 62 (01962892) , 1-16
Abstract
The semantic segmentation of remote sensing images with few shots has important theoretical and application value. Most of the existing few-shot semantic segmentation frameworks are based on prototype learning methods, in which a single support prototype is designed to guide the query set for prediction. However, the visual differences between the support set and the query set make it difficult for a single support prototype, generated from the support set, to comprehensively encapsulate the semantic information of all the query images. This paper introduces an adaptive self-supporting prototype learning network designed for few-shot segmentation (FSS), in order to tackle the challenges mentioned earlier. We propose adaptive hyper-prototype representation (HPR), which consists of hyper-prototype clustering (HPC) and guided prototype matching (GPM), to generate and assign multiple representative prototypes to compensate for the limitations of a single prototype in representing the semantic information of the query images. Specifically, HPC is a parameter-free and adaptive approach, which can extract more representative prototypes by aggregating similar feature vectors utilizing superpixel feature clustering. Meanwhile, GPM can select matched prototypes to provide more accurate guidance, allowing for uniformly aligned representation of multiple prototypes and complex image semantic information. We also introduce self-supporting matching (SSM) prototype learning, which can accurately guide the query set segmentation by acquiring query set prototypes. SSM generates initial pseudo-labels for the query set based on the support set prototypes, and further guides the query set using the pseudo-labels, along with the query prototypes generated by its own features, thus effectively avoiding visual differences between the support set and query set. The proposed adaptive self-supporting prototype learning network substantially improves the prototype quality and achieves a superior performance on object-level remote sensing datasets.Keywords
Funding Information
- National Natural Science Foundation of China (42171336, U22A20567)
This publication has 41 references indexed in Scilit:
- Semi-supervised knowledge distillation framework for global-scale urban man-made object remote sensing mappingInternational Journal of Applied Earth Observation and Geoinformation, 2023
- E2EVAP: End-to-end vectorization of smallholder agricultural parcel boundaries from high-resolution remote sensing imageryISPRS Journal of Photogrammetry and Remote Sensing, 2023
- Cross-sensor domain adaptation for high spatial resolution urban land-cover mapping: From airborne to spaceborne imageryRemote Sensing of Environment, 2022
- Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders2019
- Task Agnostic Meta-Learning for Few-Shot Learning2019
- Meta-Transfer Learning for Few-Shot Learning2019
- Superpixel Sampling Networks2018
- DOTA: A Large-Scale Dataset for Object Detection in Aerial Images2018
- Pyramid Scene Parsing Network2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015