APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation
- 11 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 25 (15209210) , 4361-4373
Abstract
Few-shot semantic segmentation aims to segment novel-class objects in a given query image with only a few labeled support images. Most advanced solutions exploit a metric learning framework that performs segmentation through matching each query feature to a learned class-specific prototype. However, this framework suffers from biased classification due to incomplete feature comparisons. To address this issue, we present an adaptive prototype representation by introducing class-specific and class-agnostic prototypes and thus construct complete sample pairs for learning semantic alignment with query features. The complementary features learning manner effectively enriches feature comparison and helps yield an unbiased segmentation model in the few-shot setting. It is implemented with a two-branch end-to-end network (\ie, a class-specific branch and a class-agnostic branch), which generates prototypes and then combines query features to perform comparisons. In addition, the proposed class-agnostic branch is simple yet effective. In practice, it can adaptively generate multiple class-agnostic prototypes for query images and learn feature alignment in a self-contrastive manner. Extensive experiments on PASCAL-5 $^i$ and COCO-20 $^i$ demonstrate the superiority of our method. At no expense of inference efficiency, our model achieves state-of-the-art results in both 1-shot and 5-shot settings for few-shot semantic segmentation.
Keywords
All Related Versions
Funding Information
- Tencent
- Special Foundation for the Development of Strategic Emerging Industries of Shenzen Municipality (JCYJ20170817161056260)
- Tencent YouTu Lab
This publication has 40 references indexed in Scilit:
- SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles2016
- Colorful Image Colorization2016
- Context Encoders: Feature Learning by Inpainting2016
- Deep Residual Learning for Image Recognition2016
- FaceNet: A unified embedding for face recognition and clustering2015
- Fully convolutional networks for semantic segmentation2015
- Spatial Pyramid Pooling in Deep Convolutional Networks for Visual RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Simultaneous Detection and Segmentation2014
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009