Lightweight Class Incremental Semantic Segmentation Without Catastrophic Forgetting
- 1 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34, 4566-4579
Abstract
Class incremental semantic segmentation (CISS) aims to progressively segment newly introduced classes while preserving the memory of previously learned ones. Traditional CISS methods directly employ advanced semantic segmentation models (e.g., Deeplab-v3) as continual learners. However, these methods require substantial computational and memory resources, limiting their deployment on edge devices. In this paper, we propose a Lightweight Class Incremental Semantic Segmentation (LISS) model tailored for resource-constrained scenarios. Specifically, we design an automatic knowledge-preservation pruning strategy based on the Hilbert-Schmidt Independence Criterion (HSIC) Lasso, which automatically compresses the CISS model by searching for global penalty coefficients. Nonetheless, reducing model parameters exacerbates catastrophic forgetting during incremental learning. To mitigate this challenge, we develop a clustering-based pseudo labels generator to obtain high-quality pseudo labels by considering the feature space structure of old classes. It adjusts predicted probabilities from the old model according to the feature proximity to nearest sub-cluster centers for each class. Additionally, we introduce a customized soft labels module that distills the semantic relationships between classes separately. It decomposes soft labels into target probabilities, background probabilities, and other probabilities, thereby maintaining knowledge of previously learned classes in a fine-grained manner. Extensive experiments on two benchmark datasets demonstrate that our LISS model outperforms state-of-the-art approaches in both effectiveness and efficiency.Keywords
Funding Information
- National Key Research and Development Program of China (2023YFB4704800)
- National Natural Science Foundation of China (62225310, 62127807)
This publication has 58 references indexed in Scilit:
- Background Adaptation with Residual Modeling for Exemplar-Free Class-Incremental Semantic Segmentation2024
- Mitigating Background Shift in Class-Incremental Semantic Segmentation2024
- Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation2024
- SATS: Self-attention transfer for continual semantic segmentationPattern Recognition, 2023
- Deep semantic segmentation of natural and medical images: a reviewArtificial Intelligence Review, 2020
- Channel pruning based on mean gradient for accelerating Convolutional Neural NetworksSignal Processing, 2019
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- ImageNet: A large-scale hierarchical image database2009
- Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem1989