DIM: long-tailed object detection and instance segmentation via dynamic instance memory
Open Access
- 23 August 2023
- journal article
- research article
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 4 (3) , 035047
Abstract
Object detection and instance segmentation have been successful on benchmarks with relatively balanced category distribution (e.g. MSCOCO). However, state-of-the-art object detection and segmentation methods still struggle to generalize on long-tailed datasets (e.g. LVIS), where a few classes (head classes) dominate the instance samples, while most classes (tailed classes) have only a few samples. To address this challenge, we propose a plug-and-play module within the Mask R-CNN framework called dynamic instance memory (DIM). Specifically, we augment Mask R-CNN with an auxiliary branch for training. It maintains a dynamic memory bank storing an instance-level prototype representation for each category, and shares the classifier with the existing instance branch. With a simple metric loss, the representations in DIM can be dynamically updated by the instance proposals in the mini-batch during training. Our DIM introduces a bias toward tailed classes to the classifier learning along with a class frequency reversed sampler, which learns generalizable representations from the original data distribution, complementing the existing instance branch. Comprehensive experiments on LVIS demonstrate the effectiveness of DIM, as well as the significant advantages of DIM over the baseline Mask R-CNN.Keywords
This publication has 14 references indexed in Scilit:
- LABNet: Local graph aggregation network with class balanced loss for vehicle re-identificationNeurocomputing, 2021
- Exploring the auxiliary learning for long-tailed visual recognitionNeurocomputing, 2021
- DSPNet: A low computational-cost network for human pose estimationNeurocomputing, 2021
- Collaborative Generative Adversarial Network with Visual perception and memory reasoningNeurocomputing, 2020
- A two-stage attention aware method for train bearing shed oil inspection based on convolutional neural networksNeurocomputing, 2020
- Deep learning-based methods for person re-identification: A comprehensive reviewNeurocomputing, 2019
- Multiview 3D human pose estimation using improved least-squares and LSTM networksNeurocomputing, 2019
- A systematic study of the class imbalance problem in convolutional neural networksNeural Networks, 2018
- Selective Search for Object RecognitionInternational Journal of Computer Vision, 2013
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009