Consensus Synergizes With Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes
- 28 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (2) , 1086-1097
Abstract
Anomaly segmentation is a critical task for safety-critical applications, such as autonomous driving in urban environments. Its objective is to detect out-of-distribution (OOD) samples with unseen categories, given a pre-trained segmentation model. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed in previous research. In this paper, we propose a simple yet effective approach named CosMe (Consensus Synergizes with Memory) to address this challenge. CosMe consists of two key components: 1) building a memory bank comprising seen prototypes extracted from multiple layers of the given segmentation model, and 2) training an auxiliary model that mimics the behavior of the given model and using the consensus of their mid-level features as complementary cues that synergize with the memory bank. The former serves as a baseline that can detect all potential outliers, including both OOD and hard in-distribution samples; the latter assists in distinguishing between these two types of outliers. Experimental results on several urban scene anomaly segmentation datasets demonstrate that CosMe outperforms previous approaches by a significant margin.Keywords
All Related Versions
Funding Information
- NSFC (62176159)
- Natural Science Foundation of Shanghai Municipality (21ZR1432200)
- Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102)
- Fundamental Research Funds for the Central Universities
This publication has 35 references indexed in Scilit:
- Lost and Found: detecting small road hazards for self-driving vehicles2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Fully convolutional networks for semantic segmentation2015
- Microsoft COCO: Common Objects in Context2014
- Toward Open Set RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Real-time object classification and novelty detection for collaborative video surveillance2003
- LOF2000