Hosub Yoon

In typical scenarios in which parts of the face are occluded by objects such as masks, sunglasses, hats, or scarves, previous studies have focused on analyzing how the recognition rate varies depending on the degree and location of occlusion, and several approaches have been proposed for detecting occluded facial areas and recognizing partial faces. However, in this study, we aim to determine which facial regions are the most crucial for occluded face recognition. Identifying in advance which features of a partial face influence recognition performance the most could help prediction and enhance recognition accuracy. To evaluate performance based on the direction and position of the occluded areas, three common deep learning‐based face recognition models (ArcFace, MobileFaceNet, and iResNet) are compared using the well‐known public face datasets, LFW, CFP‐FP, AgeDB, and IJB‐C. Extensive experiments confirmed that the eye‐centered horizontal and nose‐centered vertical regions are the most critical for face recognition. When recognition was performed using only these two regions as input, the models achieved high recognition accuracy despite the absence of other facial features.

The increase in elderly population is emerging as a serious social issue. The coronavirus pandemic has increased the number of elderly people suffering from depression and loneliness owing to the lack of face-to-face activities. In this study, we developed an integrated system for the human-care robot service, considering cognitive and emotional support for elderly people, and verified its stability and usefulness in the real world. We recruited 40 elderly people for an apartment testbed environment experiment and two elderly people living alone for a long time participated in the experiment at their homes. Quantitative experimental results were analyzed by comparing service success rates and user satisfaction in two different test environments to verify the stability of the service. Qualitative evaluations were also conducted through surveys and interviews to assess the usefulness of the service.

Face parsing, the segmentation of facial components at the pixel level, is pivotal for comprehensive facial analysis. However, previous studies encountered challenges, showing reduced performance in areas with small or thin classes like necklaces and earrings, and struggling to adapt to occlusion scenarios such as masks, glasses, caps or hands. To address these issues, this study proposes a robust face parsing technique through the strategic integration of self-attention and self-distillation methods. The self-attention module enhances contextual information, enabling precise feature identification for each facial element. Multi-task learning for edge detection, coupled with a specialized loss function focusing on edge regions, elevates the understanding of fine structures and contours. Additionally, the application of self-distillation for fine-tuning proves highly efficient, producing refined parsing results while maintaining high performance in scenarios with limited labels and ensuring robust generalization. The integration of self-attention and self-distillation techniques addresses challenges of previous studies, particularly in handling small or thin classes. This strategic fusion enhances overall performance, achieving computational efficiency, and aligns with the latest trends in this research area. The proposed approach attains a Mean F1 score of 88.18% on the CelebAMask-HQ dataset, marking a significant advancement in face parsing with state-of-the-art performance. Even in challenging occlusion areas like hands and masks, it demonstrates a remarkable F1 score of over 99%, showcasing robust face parsing capabilities in real-world environments.

Image-based facial identity recognition has become a technology that is now used in many applications. This is because it is possible to use only a camera without the need for any other device. Besides, due to the advantage of contactless technology, it is one of the most popular certifications. However, a common recognition system is not possible if some of the face information is lost due to the user’s posture or the wearing of masks, as caused by the recent prevalent disease. In some platforms, although performance is improved through incremental updates, it is still inconvenient and inaccurate. In this paper, we propose a method to respond more actively to these situations. First, we determine whether an obscurity occurs and improve the stability by calculating the feature vector using only a significant area when the obscurity occurs. By recycling the existing recognition model, without incurring little additional costs, the results of reducing the recognition performance drop in certain situations were confirmed. Using this technique, we confirmed a performance improvement of about 1~3% in a situation where some information is lost. Although the performance is not dramatically improved, it has the big advantage that it can improve recognition performance by utilizing existing systems.

We described a real-time hair segmentation method based on a fully convolutional network with the basic structure of an encoder–decoder. In one of the traditional computer vision techniques for hair segmentation, the mean shift and watershed methodologies suffer from inaccuracy and slow execution due to multi-step, complex image processing. It is also difficult to execute the process in real-time unless an optimization technique is applied to the partition. To solve this problem, we exploited Mobile-Unet using the U-Net segmentation model, which incorporates the optimization techniques of MobileNetV2. In experiments, hair segmentation accuracy was evaluated by different genders and races, and the average accuracy was 89.9%. By comparing the accuracy and execution speed of our model with those of other models in related studies, we confirmed that the proposed model achieved the same or better performance. As such, the results of hair segmentation can obtain hair information (style, color, length), which has a significant impact on human-robot interaction with people.

Facial landmark localization is essential for robot-human interaction. In particular, the human eye is more important because it can grasp a person’s interests. However, the traditional method does not consider eye changes from the dataset, so the limitation is clear, this paper presents a data augmentation method for acquiring various eye images and a method for creating a robust eye landmark model with 2-stage training. Experiments on augmented 300W-LP datasets show that our method outperforms performance than the previous method.