Rethinking Obscured Sub-Optimality in Analytic Learning for Exemplar-Free Class-Incremental Learning
- 11 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (4) , 4287-4301
Abstract
Exemplar-free Class-Incremental Learning (EFCIL) poses a significant challenge in mitigating catastrophic forgetting, due to the absence of exemplars. Recently, analytic learning-based methods propose a recursive alignment procedure to execute EFCIL in a phase-invariant manner and show state-of-the-art performance. However, they heavily rely on a frozen feature extractor trained with the initial dataset to avoid the misalignment between feature and label spaces, ignoring the importance of acquiring generalizable features across incremental tasks for performance improvement. To tackle this, we rethink the obscured sub-optimality of analytic learning-based methods, particularly through empirical reevaluation, and then introduce the Multi-head analytic learning (Muheal) approach. Muheal forms the multi-head model with a delicate feature extractor, thereby introducing a feature optimization procedure and a forgetting compensation module to balance the learning and forgetting. Specifically, within the feature optimization procedure, the feature extractor seeks to learn more generalizable features in a self-supervised manner using the fully-connected classification head. An analytic learning-based classification head follows to align the feature-label space. Additionally, we employ the compensation module to generate and align pseudo-features with a replicated analytic head, thus preventing overfitting and testing. Comprehensive experiments on several benchmark datasets have demonstrated that Muheal significantly outperforms existing state-of-the-art EFCIL methods and is comparable, if not superior, to methods that use replay techniques.Keywords
Funding Information
- National Science and Technology Major Project (2023ZD0121101)
- National Natural Science Foundation of China (62172426)
- National Natural Science Foundation of China (ZZCX-ZZGC-01-04)
This publication has 43 references indexed in Scilit:
- Less confidence, less forgetting: Learning with a humbler teacher in exemplar-free Class-Incremental learningNeural Networks, 2024
- PyCIL: a Python toolbox for class-incremental learningScience China Information Sciences, 2023
- FOSTER: Feature Boosting and Compression for Class-Incremental Learning2022
- Correlation Projection for Analytic Learning of a Classification NetworkNeural Processing Letters, 2021
- PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning2020
- Continual lifelong learning with neural networks: A reviewNeural Networks, 2019
- Deep Residual Learning for Image Recognition2016
- Consciousness CLEARS the mindNeural Networks, 2007
- A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth dataNeurocomputing, 2004
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989