Mask-Guided Deformation Adaptive Network for Human Parsing
Open Access
- 31 January 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 18 (1) , 1-20
Abstract
Due to the challenges of densely compacted body parts, nonrigid clothing items, and severe overlap in crowd scenes, human parsing needs to focus more on multilevel feature representations compared to general scene parsing tasks. Based on this observation, we propose to introduce the auxiliary task of human mask and edge detection to facilitate human parsing. Different from human parsing, which exploits the discriminative features of each category, human mask and edge detection emphasizes the boundaries of semantic parsing regions and the difference between foreground humans and background clutter, which benefits the parsing predictions of crowd scenes and small human parts. Specifically, we extract human mask and edge labels from the human parsing annotations and train a shared encoder with three independent decoders for the three mutually beneficial tasks. Furthermore, the decoder feature maps of the human mask prediction branch are further exploited as attention maps, indicating human regions to facilitate the decoding process of human parsing and human edge detection. In addition to these auxiliary tasks, we further alleviate the problem of deformed clothing items under various human poses by tracking the deformation patterns with the deformable convolution. Extensive experiments show that the proposed method can achieve superior performance against state-of-the-art methods on both single and multiple human parsing datasets. Codes and trained models are available https://github.com/ViktorLiang/MGDAN .Keywords
Funding Information
- National Natural Science Foundation of China (61972162)
- Guangdong International Science and Technology Cooperation Project (2021A0505030009)
- Guangdong Natural Science Foundation (2019A1515010833, 2021A1515012625)
- Guangzhou Basic and Applied Research Project (202102021074)
- Fundamental Research Funds for the Central Universities (2020ZYGXZR089)
- Social Science Research Base of Guangdong Province-Research Center of Network Civilization in New Era of SCUT
- CCF-Tencent Open Research Fund (CCF-Tencent RAGR20190112)
This publication has 42 references indexed in Scilit:
- Self-Supervised Neural Aggregation Networks for Human Parsing2017
- Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform2016
- Clothes Co-Parsing Via Joint Image Segmentation and Labeling With Application to Clothing RetrievalIEEE Transactions on Multimedia, 2016
- Multimodal Multipart Learning for Action Recognition in Depth VideosIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Matching-CNN meets KNN: Quasi-parametric human parsing2015
- Deep Human Parsing with Active Template RegressionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Retrieving Similar Styles to Parse ClothingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Detect What You Can: Detecting and Representing Objects Using Holistic Models and Body Parts2014
- Fashion Parsing With Weak Color-Category LabelsIEEE Transactions on Multimedia, 2013
- Object Detection with Discriminatively Trained Part-Based ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009