LightSeg: Local Spatial Perception Convolution for Real-Time Semantic Segmentation
Open Access
- 12 July 2023
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 13 (14) , 8130
Abstract
Semantic segmentation is increasingly being applied on mobile devices due to advancements in mobile chipsets, particularly in low-power consumption scenarios. However, the lightweight design of mobile devices poses limitations on the receptive field, which is crucial for dense prediction problems. Existing approaches have attempted to balance lightweight designs and high accuracy by downsampling features in the backbone. However, this downsampling may result in the loss of local details at each network stage. To address this challenge, this paper presents a novel solution in the form of a compact and efficient convolutional neural network (CNN) for real-time applications: our proposed model, local spatial perception convolution (LSPConv). Furthermore, the effectiveness of our architecture is demonstrated on the Cityscapes dataset. The results show that our model achieves an impressive balance between accuracy and inference speed. Specifically, our LightSeg, which does not rely on ImageNet pretraining, achieves an mIoU of 76.1 at a speed of 61 FPS on the Cityscapes validation set, utilizing an RTX 2080Ti GPU with mixed precision. Additionally, it achieves a speed of 115.7 FPS on the Jetson NX with int8 precision.Keywords
Funding Information
- National Natural Science Foundation of China (61876049, 62172118, 2021GXNSFDA196002, 2021YFQ0002, GIIP2004, S202210595177, 202210595029)
- Nature Science key Foundation of Guangxi (61876049, 62172118, 2021GXNSFDA196002, 2021YFQ0002, GIIP2004, S202210595177, 202210595029)
- Sichuan Regional Innovation Cooperation Project (61876049, 62172118, 2021GXNSFDA196002, 2021YFQ0002, GIIP2004, S202210595177, 202210595029)
- Guangxi Key Laboratory of Image and Graphic Intelligent Processing (61876049, 62172118, 2021GXNSFDA196002, 2021YFQ0002, GIIP2004, S202210595177, 202210595029)
- Student’s Platform for Innovation and Entrepreneurship Training Program (61876049, 62172118, 2021GXNSFDA196002, 2021YFQ0002, GIIP2004, S202210595177, 202210595029)
This publication has 30 references indexed in Scilit:
- BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation2018
- Understanding Convolution for Semantic Segmentation2018
- Dilated Residual Networks2017
- Aggregated Residual Transformations for Deep Neural Networks2017
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Fully convolutional networks for semantic segmentation2015
- Semantic object classes in video: A high-definition ground truth databasePattern Recognition Letters, 2009
- Segmentation and Recognition Using Structure from Motion Point Clouds2008