Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
Authors:
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8721-8730
Abstract
A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels). Most existing methods meta-learn all three model components for fast adaptation to a new class. However, given that as few as a single support set image is available, effective model adaption of all three components to the new class is extremely challenging. In this work we propose to simplify the meta-learning task by focusing solely on the simplest component – the classifier, whilst leaving the en-coder and decoder to pre-training. We hypothesize that if we pretrain an off-the-shelf segmentation model over a set of diverse training classes with sufficient annotations, the encoder and decoder can capture rich discriminative features applicable for any unseen classes, rendering the sub-sequent meta-learning stage unnecessary. For the classifier meta-learning, we introduce a Classifier Weight Transformer (CWT) designed to dynamically adapt the support-set trained classifier’s weights to each query image in an inductive way. Extensive experiments on two standard bench-marks show that despite its simplicity, our method outperforms the state-of-the-art alternatives, often by a large margin. Code is available on https://github.com/zhiheLu/CWT-for-FSS.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation2019
- Feature Weighting and Boosting for Few-Shot Segmentation2019
- CANet: Class-Agnostic Segmentation Networks With Iterative Refinement and Attentive Few-Shot Learning2019
- Relation Networks for Object Detection2018
- Dynamic Few-Shot Visual Learning Without Forgetting2018
- Pyramid Scene Parsing Network2017
- Deep Residual Learning for Image Recognition2016
- Fully convolutional networks for semantic segmentation2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009