RTMDet-R: A Robust Instance Segmentation Network for Complex Traffic Scenarios
- 17 February 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (6) , 8834-8847
Abstract
In complex traffic scenarios, several factors including lighting, weather, the size of the traffic participants, the distance between the traffic participants and the camera, and occlusions impact the features of the traffic participants. The impact of these factors is a huge challenge, especially for vision-based instance segmentation networks. To this end, this paper proposes an enhanced version of the RTMDet to promote the overall performance of instance segmentation in complex traffic scenarios. Firstly, an extended CSP-style backbone with large kernel convolutions of different kernel sizes is used to enhance the robustness of feature extraction capability, which contributes to obtaining more information about traffic objects of different scales. Secondly, a plugin pre-fusion module is designed to enhance the network’s robustness to multi-scale changes caused by distance changes. Additionally, instance kernel distinguish module is proposed to further highlight and distinguish different instance objects under poor lighting or weather and occlusion situations. Finally, the existing advanced image generation technology is used to expand the BDD100k dataset, enriching the dataset with severe scenarios. With an input resolution of $\mathbf{1280}\times\mathbf{720}$ on the expanded BDD100K dataset, the proposed RTMDet-R achieves an accuracy of 25.4% mAP on the instance mask and 27.8% mAP on the instance box. This surpasses other similar models in terms of accuracy. Additionally, it maintains a good inference speed of 23.1 FPS, achieving the trade-off between accuracy and speed. Code and models are released at https://github.com/GTrui6/RTMDet-R.git.Keywords
Funding Information
- National Key Research and Development Program of China (2023YFB2504400)
- National Natural Science Foundation of China (52225212, U20A20333, 52072160)
This publication has 45 references indexed in Scilit:
- Visual attention networkComputational Visual Media, 2023
- Conditional Convolutions for Instance Segmentation2020
- Densely Connected Convolutional Networks2017
- Feature Pyramid Networks for Object Detection2017
- Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network2017
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation2016
- SSD: Single Shot MultiBox Detector2016
- Deep Residual Learning for Image Recognition2016
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Microsoft COCO: Common Objects in Context2014