Deep Spectral Improvement for Unsupervised Image Instance Segmentation
Preprint
Abstract
Deep spectral methods reframe the image decomposition process as a graph partitioning task by extracting features using self-supervised learning and utilizing the Laplacian of the affinity matrix to obtain eigensegments. However, instance segmentation has received less attention compared to other tasks within the context of deep spectral methods. This paper addresses the fact that not all channels of the feature map extracted from a self-supervised backbone contain sufficient information for instance segmentation purposes. In fact, Some channels are noisy and hinder the accuracy of the task. To overcome this issue, this paper proposes two channel reduction modules: Noise Channel Reduction (NCR) and Deviation-based Channel Reduction (DCR). The NCR retains channels with lower entropy, as they are less likely to be noisy, while DCR prunes channels with low standard deviation, as they lack sufficient information for effective instance segmentation. Furthermore, the paper demonstrates that the dot product, commonly used in deep spectral methods, is not suitable for instance segmentation due to its sensitivity to feature map values, potentially leading to incorrect instance segments. A new similarity metric called Bray-Curtis over Chebyshev (BoC) is proposed to address this issue. It takes into account the distribution of features in addition to their values, providing a more robust similarity measure for instance segmentation. Quantitative and qualitative results on the Youtube-VIS2019 dataset highlight the improvements achieved by the proposed channel reduction methods and the use of BoC instead of the conventional dot product for creating the affinity matrix. These improvements are observed in terms of mean Intersection over Union and extracted instance segments, demonstrating enhanced instance segmentation performance. The code is available on: https://github.com/farnooshar/SpecUnIISKeywords
All Related Versions
This publication has 10 references indexed in Scilit:
- TokenCut: Segmenting Objects in Images and Videos With Self-Supervised Transformer and Normalized CutIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- Occluded Video Instance Segmentation: A BenchmarkInternational Journal of Computer Vision, 2022
- Localizing Objects with Self-supervised Transformers and no Labels2021
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- Unsupervised Visual Representation Learning by Context Prediction2015
- The Pascal Visual Object Classes Challenge: A RetrospectiveInternational Journal of Computer Vision, 2014
- Normalized cuts and image segmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2000
- Lower Bounds for the Partitioning of GraphsIBM Journal of Research and Development, 1973
- An Ordination of the Upland Forest Communities of Southern WisconsinEcological Monographs, 1957