Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery
Authors:
Preprint
Abstract
Discovering novel concepts from unlabelled data and in a continuous manner is an important desideratum of lifelong learners. In the literature such problems have been partially addressed under very restricted settings, where either access to labelled data is provided for discovering novel concepts (e.g., NCD) or learning occurs for a limited number of incremental steps (e.g., class-iNCD). In this work we challenge the status quo and propose a more challenging and practical learning paradigm called MSc-iNCD, where learning occurs continuously and unsupervisedly, while exploiting the rich priors from large-scale pre-trained models. To this end, we propose simple baselines that are not only resilient under longer learning scenarios, but are surprisingly strong when compared with sophisticated state-of-the-art methods. We conduct extensive empirical evaluation on a multitude of benchmarks and show the effectiveness of our proposed baselines, which significantly raises the bar.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Novel Class Discovery Without Forgetting2022
- Class-Incremental Novel Class Discovery2022
- Transfer Without Forgetting2022
- Neighborhood Contrastive Learning for Novel Class Discovery2021
- OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World2021
- Learning a Unified Classifier Incrementally via Rebalancing2019
- Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks2018
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- Joint Unsupervised Learning of Deep Representations and Image Clusters2016
- Catastrophic Forgetting in Connectionist Networks2006