Portrait Semantic Segmentation Method Based on Dual Modal Information Complementarity
Open Access
- 9 February 2024
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 14 (4) , 1439
Abstract
Semantic segmentation of human images is a research hotspot in the field of computer vision. At present, the semantic segmentation models based on U-net generally lack the ability to capture the spatial information of images. At the same time, semantic incompatibility exists because the feature maps of encoder and decoder are directly connected in the skip connection stage. In addition, in low light scenes such as at night, it is easy for false segmentation and segmentation accuracy to appear. To solve the above problems, a portrait semantic segmentation method based on dual-modal information complementarity is proposed. The encoder adopts a double branch structure, and uses a SK-ASSP module that can adaptively adjust the convolution weights of different receptor fields to extract features in RGB and gray image modes respectively, and carries out cross-modal information complementarity and feature fusion. A hybrid attention mechanism is used in the jump connection phase to capture both the channel and coordinate context information of the image. Experiments on human matting dataset show that the PA and MIoU coefficients of this algorithm model reach 96.58% and 94.48% respectively, which is better than U-net benchmark model and other mainstream semantic segmentation models.Keywords
Funding Information
- Guangdong Provincial Philosophy and Social Science Planning Project of China (GD23YJY08)
This publication has 18 references indexed in Scilit:
- SCAttNet: Semantic Segmentation Network With Spatial and Channel Attention Mechanism for High-Resolution Remote Sensing ImagesIEEE Geoscience and Remote Sensing Letters, 2020
- Building Extraction Based on U-Net with an Attention Block and Multiple LossesRemote Sensing, 2020
- Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks2019
- Self-Supervised Model Adaptation for Multimodal Semantic SegmentationInternational Journal of Computer Vision, 2019
- Dual Attention Network for Scene Segmentation2019
- PortraitNet: Real-time portrait segmentation network for mobile deviceComputers & Graphics, 2019
- Semantic Human Matting2018
- CBAM: Convolutional Block Attention Module2018
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation2018
- LinkNet: Exploiting encoder representations for efficient semantic segmentation2017