Multimodal Framework for Long-Tailed Recognition
Open Access
- 16 November 2024
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 14 (22) , 10572
Abstract
Long-tailed data distribution (i.e., minority classes occupy most of the data, while most classes have very few samples) is a common problem in image classification. In this paper, we propose a novel multimodal framework for long-tailed data recognition. In the first stage, long-tailed data are used for visual-semantic contrastive learning to obtain good features, while in the second stage, class-balanced data are used for classifier training. The proposed framework leverages the advantages of multimodal models and mitigates the problem of class imbalance in long-tailed data recognition. Experimental results demonstrate that the proposed framework achieves competitive performance on the CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist2018 datasets for image classification.Keywords
This publication has 23 references indexed in Scilit:
- A Long-Tail Relation Extraction Model Based on Dependency Path and Relation Graph Embedding2024
- Joint representation and classifier learning for long-tailed image classificationImage and Vision Computing, 2023
- Deep Long-Tailed Learning: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Long-tail Detection with Effective Class-Margins2022
- Learning to Prompt for Vision-Language ModelsInternational Journal of Computer Vision, 2022
- Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-Tailed Classification2020
- Remix: Rebalanced Mixup2020
- AutoAugment: Learning Augmentation Strategies From Data2019
- Class-Balanced Loss Based on Effective Number of Samples2019
- Empirical Risk Minimization2017