A Lightweight Model for Fast and Detail-Preserving Human Segmentation in Video
- 1 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 10 (1) , 91-104
Abstract
With the rapid development of artificial intelligence, human segmentation in video is becoming increasingly important in the field of computer vision. However, existing segmentation models suffer from inaccurate segmentation, slow processing, and large model size that limit their deployment on resource-constrained devices. To this end, we propose a lightweight model called Efficient Memory Aggregation U-shaped Network (EMAUnet) for human segmentation in video, which is based on a traditional U-shaped network and attention mechanism. In EMAUnet, memory modules are combined with segmentation modules, enabling end-to-end learning of semantic extraction patterns for human images. Mobile-inverse bottleneck convolution (MBConv) is used as the network backbone that has relatively few parameters and computational complexity. Inverted sub-pixel down-sampling (ISP) is proposed to minimize information loss and achieve detail-preserving of segmentation. Coordinate attention (CA) is adopted to precisely locate the portrait area. Moreover, bidirectional memory update (BMU) and memory update trigger (MUT) are proposed to improve memory resource utilization and reduce unnecessary computation. Experimental results show that, compared with the classic model ISNet, EMAUnet has the mIoU, FPS and pixel accuracy increased by 2.3%, 25.0% and 1.8%, respectively, while the amount of parameters and the size of model decreased by 33.3% and 45.7%, respectively.Keywords
Funding Information
- National Natural Science Foundation of China (62172450)
- Key R&D plan of Hunan Province (2022GK2008)
- Fundamental Research Funds for Central Universities of the Central South University (1053320241713)
This publication has 45 references indexed in Scilit:
- ISM-Net: Mining incremental semantics for class incremental learningNeurocomputing, 2022
- Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation2022
- Video Instance Segmentation of Rock Particle Based on MaskTrack R-CNN2022
- BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic SegmentationInternational Journal of Computer Vision, 2021
- Fast Video Object Segmentation Using the Global Context Module2020
- U2-Net: Going deeper with nested U-structure for salient object detectionPattern Recognition, 2020
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design2018
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018
- A survey on deep learning techniques for image and video semantic segmentationApplied Soft Computing, 2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015