MiSSNet: Memory-Inspired Semantic Segmentation Augmentation Network for Class-Incremental Learning in Remote Sensing Images

Authors:
Publisher website
Google Scholar
Abstract
With remote sensing images constantly being collected rapidly, class-incremental semantic segmentation task has attracted increasing attention. However, the semantic distribution shift problem of the background class in remote sensing images, which is a case of catastrophic forgetting, continues to limit available class-incremental semantic segmentation algorithms. To address this challenge, we present a new Memory-inspired Semantic Segmentation augmentation network (MiSSNet) for class-incremental learning in remote sensing images. The MiSSNet mainly includes two modules: Local Semantic Distillation (LSD) module and Class-Specific Regularization (CSR) module. LSD is a distillation structure that employs the local semantic features in retained memory to maintain correlation between pixels throughout the training process of incremental learning. It constructs a series of pixel-level correlation matrices and implicitly adjusts the semantic distribution shift problem of the background class. CSR is a class-wise regularization term that utilizes the class-specific portion of the preserved memory to help the model keep repeating the learning of the old categories. It alleviates the background classes shift problem by generating countless pixel level instances of old classes. LSD and CSR work together to tackle the semantic distribution shift problem of background class from semantic information and class information aspects, respectively. Specially, MiSSNet only needs additional single inference process for memory extraction and storage, and the whole algorithm does not add any new training parameters. Experimental results on three semantic segmentation datasets indicate the advantage of the proposed method.
Funding Information
  • National Key Research and Development Program of China (2022YFA1003800, 2022ZD0160401)
  • National Natural Science Foundation of China (62001251, 62001252, 62125102)
  • Beijing–Tianjin–Hebei Basic Research Cooperation Project (F2021203109)

This publication has 56 references indexed in Scilit: