Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer
Preprint
Abstract
A few-shot semantic segmentation model is typically composed of a CNN encoder, a CNN decoder and a simple classifier (separating foreground and background pixels). Most existing methods meta-learn all three model components for fast adaptation to a new class. However, given that as few as a single support set image is available, effective model adaption of all three components to the new class is extremely challenging. In this work we propose to simplify the meta-learning task by focusing solely on the simplest component, the classifier, whilst leaving the encoder and decoder to pre-training. We hypothesize that if we pre-train an off-the-shelf segmentation model over a set of diverse training classes with sufficient annotations, the encoder and decoder can capture rich discriminative features applicable for any unseen classes, rendering the subsequent meta-learning stage unnecessary. For the classifier meta-learning, we introduce a Classifier Weight Transformer (CWT) designed to dynamically adapt the supportset trained classifier's weights to each query image in an inductive way. Extensive experiments on two standard benchmarks show that despite its simplicity, our method outperforms the state-of-the-art alternatives, often by a large margin.Code is available on https://github.com/zhiheLu/CWT-for-FSS.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Few-Shot Semantic Segmentation with Democratic Attention Networks2020
- Rethinking Few-Shot Image Classification: A Good Embedding is All You Need?2020
- When Does Self-supervision Improve Few-Shot Learning?2020
- Prototype Mixture Models for Few-Shot Semantic Segmentation2020
- Part-Aware Prototype Network for Few-Shot Semantic Segmentation2020
- Prototype Rectification for Few-Shot Learning2020
- End-to-End Object Detection with Transformers2020
- Negative Margin Matters: Understanding Margin in Few-Shot Classification2020
- Microsoft COCO: Common Objects in Context2014
- Simultaneous Detection and Segmentation2014