Multi-Granularity Part Sampling Attention for Fine-Grained Visual Classification
- 16 August 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 33 (10577149) , 4529-4542
Abstract
Fine-grained visual classification aims to classify similar sub-categories with the challenges of large variations within the same sub-category and high visual similarities between different sub-categories. Recently, methods that extract semantic parts of the discriminative regions have attracted increasing attention. However, most existing methods extract the part features via rectangular bounding boxes by object detection module or attention mechanism, which makes it difficult to capture the rich shape information of objects. In this paper, we propose a novel Multi-Granularity Part Sampling Attention (MPSA) network for fine-grained visual classification. First, a novel multi-granularity part retrospect block is designed to extract the part information of different scales and enhance the high-level feature representation with discriminative part features of different granularities. Then, to extract part features of various shapes at each granularity, we propose part sampling attention, which can sample the implicit semantic parts on the feature maps comprehensively. The proposed part sampling attention not only considers the importance of sampled parts but also adopts the part dropout to reduce the overfitting issue. In addition, we propose a novel multi-granularity fusion method to highlight the foreground features and suppress the background noises with the assistance of the gradient class activation map. Experimental results demonstrate that the proposed MPSA achieves state-of-the-art performance on four commonly used fine-grained visual classification benchmarks. The source code is publicly available at https://github.com/mobulan/MPSA.Keywords
Funding Information
- Anhui Provincial Key Research and Development Program (2022i01020014)
- Natural Science Foundation for the Higher Education Institutions of Anhui Province (KJ2021A0038)
- Natural Science Foundation of Anhui Province (2108085Y23)
- National Natural Science Foundation of China (61860206004)
- University Synergy Innovation Program of Anhui Province (GXXT-2022-032)
This publication has 59 references indexed in Scilit:
- SwinFG: A fine-grained recognition scheme based on swin transformerExpert Systems with Applications, 2023
- Accurate Fine-Grained Object Recognition with Structure-Driven Relation Graph NetworksInternational Journal of Computer Vision, 2023
- Granularity-aware distillation and structure modeling region proposal network for fine-grained image classificationPattern Recognition, 2023
- Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction2022
- Improving Fine-Grained Visual Recognition in Low Data Regimes via Self-boosting Attention Mechanism2022
- Part-based annotation-free fine-grained classification of images of retail productsPattern Recognition, 2022
- Domain Adaptive Transfer Learning on Visual Attention Aware Data Augmentation for Fine-Grained Visual Categorization2020
- Fine-Grained Visual Classification via Progressive Multi-granularity Training of Jigsaw Patches2020
- Fine-Grained Person Re-identificationInternational Journal of Computer Vision, 2020
- Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorizationPattern Recognition, 2018