Robust Feature Learning and Global Variance-Driven Classifier Alignment for Long-Tail Class Incremental Learning
Preprint
Abstract
This paper introduces a two-stage framework designed to enhance long-tail class incremental learning, enabling the model to progressively learn new classes, while mitigating catastrophic forgetting in the context of long-tailed data distributions. Addressing the challenge posed by the under-representation of tail classes in long-tail class incremental learning, our approach achieves classifier alignment by leveraging global variance as an informative measure and class prototypes in the second stage. This process effectively captures class properties and eliminates the need for data balancing or additional layer tuning. Alongside traditional class incremental learning losses in the first stage, the proposed approach incorporates mixup classes to learn robust feature representations, ensuring smoother boundaries. The proposed framework can seamlessly integrate as a module with any class incremental learning method to effectively handle long-tail class incremental learning scenarios. Extensive experimentation on the CIFAR-100 and ImageNet-Subset datasets validates the approach's efficacy, showcasing its superiority over state-of-the-art techniques across various long-tail CIL settings.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Long-Tailed Class Incremental Learning2022
- PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning2020
- Large-Scale Long-Tailed Recognition in an Open World2019
- Learning a Unified Classifier Incrementally via Rebalancing2019
- Large Scale Incremental Learning2019
- Memory Aware Synapses: Learning What (not) to Forget2018
- Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights2018
- PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning2018
- Learning Fine-Grained Image Similarity with Deep Ranking2014
- Visual and semantic similarity in ImageNet2011