Dynamic Knowledge Adapter with Probabilistic Calibration for Generalized Few-Shot Semantic Segmentation
Authors:
- 17 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2781-2790
Abstract
Generalized Few-shot Semantic Segmentation (GFSS) aims to use a few novel-class samples to enable the model trained on base classes to have the ability to segment for all classes (including base and novel classes). We analyze the three main reasons for the model’s limited performance on GFSS: the lack of adaptability to learn novel classes, the instability that causes the catastrophic forgetting of base classes, and the biased prediction of imbalanced classes. To handle these issues, we design an auxiliary network (Dynamic Knowledge Adapter, DKA) for the GFSS task. Firstly, DKA handles the adaptability problem by selecting only efficient parameters for finetuning. Secondly, DKA addresses the stability problem by relabelling part of the training samples for iterative training, which alleviates the conflict between base and novel classes. Thirdly, it involves a probabilistic calibration module to help the model rectify the prediction bias caused by imbalanced data. Experimental results show that these designs can help the model to take into account the segmentation performance of base classes, novel classes, and the background class, that is, to perform well in all-class segmentation.Keywords
This publication has 37 references indexed in Scilit:
- Rethinking Few-Shot Image Classification: A Good Embedding is All You Need?2020
- Prototype Mixture Models for Few-Shot Semantic Segmentation2020
- Part-Aware Prototype Network for Few-Shot Semantic Segmentation2020
- ICNet for Real-Time Semantic Segmentation on High-Resolution Images2018
- PSANet: Point-wise Spatial Attention Network for Scene Parsing2018
- Learning to Compare: Relation Network for Few-Shot Learning2018
- Pyramid Scene Parsing Network2017
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Fully convolutional networks for semantic segmentation2015
- Learning from one example through shared densities on transforms2002