Enhancing Few-Shot Image Classification With Cosine Transformer
Open Access
- 24 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 79659-79672
Abstract
This paper addresses the few-shot image classification problem, where the classification task is performed on unlabeled query samples given a small amount of labeled support samples only. One major challenge of the few-shot learning problem is the large variety of object visual appearances that prevents the support samples to represent that object comprehensively. This might result in a significant difference between support and query samples, therefore undermining the performance of few-shot algorithms. In this paper, we tackle the problem by proposing Few-shot Cosine Transformer (FS-CT), where the relational map between supports and queries is effectively obtained for the few-shot tasks. The FS-CT consists of two parts, Learnable Prototypical embedding network to obtain categorical representations from support samples with hard cases, and Transformer encoder to effectively achieve the relational map from two different support and query samples. We introduce Cosine Attention, a more robust and stable attention module that enhances the transformer module significantly and therefore improves FS-CT performance from 5% to over 20% in accuracy compared to the default scaled dot-product mechanism. Our method performs comparison results in mini -ImageNet, CUB-200, and CIFAR-FS few-shot datasets under various configurations. We also developed a custom few-shot dataset for Yoga pose recognition to demonstrate the potential of our algorithm for practical application. Our FS-CT with Cosine Attention is a lightweight, straightforward few-shot algorithm that can be applied for a wide range of applications, such as healthcare, medical, and security surveillance. The official implementation code of our Few-shot Cosine Transformer is available at https://github.com/vinuni-vishc/Few-Shot-Cosine-Transformer.Keywords
All Related Versions
Funding Information
- Vingroup Joint Stock Company (Vingroup JSC), Vingroup
- Vingroup Innovation Foundation (VINIF.2021.DA00128)
This publication has 27 references indexed in Scilit:
- Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks2018
- Learning to Compare: Relation Network for Few-Shot Learning2018
- Few-Shot Image Recognition by Predicting Parameters from Activations2018
- Low-Shot Learning from Imaginary Data2018
- Dynamic Few-Shot Visual Learning Without Forgetting2018
- Local and Global Feature Learning for Blind Quality Evaluation of Screen Content and Natural Scene ImagesIEEE Transactions on Image Processing, 2018
- Deep Residual Learning for Image Recognition2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- ImageNet: A large-scale hierarchical image database2009
- One-shot learning of object categoriesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2006