Jang-Hee Yoo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jang-Hee Yoo has a total of 83 co-authors. Here are the top co-authors by number of shared publications:
- Mark S. Nixon
University of Southampton
6 shared publications Byung Jun Kang
Technical Research Institute, Hyundai Mobis, Gyeonggi, S. Korea
4 shared publicationsKang Ryoung Park
Dongguk University, Division of Electronics and Electrical Engineering, 26, Pil-dong 3-ga, Jung-gu, Seoul 100-715, Republic of Korea
4 shared publicationsKi-Young Moon
Information Security Research Division, ETRI, S. Korea
4 shared publications- Cheol-Hwan Yoo
ETRI
3 shared publications Doosung Hwang
Department of Computer Science, Dankook University, S. Korea
3 shared publications
Latest Publications
- Open Access
- 26 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 32736-32754
A Taxonomy of Knowledge Bases for Retrieval-Augmented Methods in Vision: A Comprehensive Survey
Recent vision–language models (VLMs) demonstrate impressive performance but often hallucinate when essential knowledge is not encoded in their parameters. Retrieval-augmented generation (RAG), which integrates external knowledge bases (KBs), has thus been widely adopted in computer vision. Beyond VLMs, retrieval augmentation has also been applied independently to improve task performance. However, no systematic analysis has investigated which types of KBs are most suitable for particular vision tasks. In this study, we present the first comprehensive survey of more than sixty studies (2021–2025) with a novel focus on KB types. We propose a taxonomy consisting of six categories: unstructured text (V-UT), ontology (V-OT), image (V-IM), image–text pairs (V-IT), structured graphs (V-SG), and domain-specific data (V-DM). For each category, we review retrieval pipelines, downstream tasks, representative datasets, and indexing strategies under consistent criteria. Our analysis shows that, regardless of KB type, most systems converge on dense encoders with vector databases, reflecting a mature technical stack. Nevertheless, this convergence often underutilizes KB-specific structures, highlighting significant opportunities for future studies. Finally, we provide practical guidelines for KB selection and retrieval design in vision and emphasize the need for KB-specific retrieval methods and standardized benchmarks.
- 26 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 32736-32754
Open AccessA Taxonomy of Knowledge Bases for Retrieval-Augmented Methods in Vision: A Comprehensive Survey
Recent vision–language models (VLMs) demonstrate impressive performance but often hallucinate when essential knowledge is not encoded in their parameters. Retrieval-augmented generation (RAG), which integrates external knowledge bases (KBs), has thus been widely adopted in computer vision. Beyond VLMs, retrieval augmentation has also been applied independently to improve task performance. However, no systematic analysis has investigated which types of KBs are most suitable for particular vision tasks. In this study, we present the first comprehensive survey of more than sixty studies (2021–2025) with a novel focus on KB types. We propose a taxonomy consisting of six categories: unstructured text (V-UT), ontology (V-OT), image (V-IM), image–text pairs (V-IT), structured graphs (V-SG), and domain-specific data (V-DM). For each category, we review retrieval pipelines, downstream tasks, representative datasets, and indexing strategies under consistent criteria. Our analysis shows that, regardless of KB type, most systems converge on dense encoders with vector databases, reflecting a mature technical stack. Nevertheless, this convergence often underutilizes KB-specific structures, highlighting significant opportunities for future studies. Finally, we provide practical guidelines for KB selection and retrieval design in vision and emphasize the need for KB-specific retrieval methods and standardized benchmarks.
- Open Access
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 211138-211150
Interpretable Autism Screening via Object-Based Analysis of Child-Drawn Sketches
Developmental assessment methods for children largely rely on subjective manual scoring, which leads to limitations in interpretability, consistency, and scalability. For high-stakes applications in clinical and educational environments, high accuracy and transparency are essential in decision-making, which underscores the growing importance of explainable artificial intelligence techniques for automated developmental assessment. To address these challenges, we propose an explainable deep learning framework for automated developmental assessment based on freehand portraits drawn by children. The proposed system integrates clinically evaluated criteria directly into the design of a computational model to ensure both accuracy and interpretability. Focusing specifically on unstructured freehand portrait sketches, the framework introduces an object-based contribution analysis that segments each sketch into semantically meaningful parts such as eyes, nose, ears, and torso, for localized contribution scoring. A customized vision transformer architecture equipped with relative position encoding and safe layer normalization was utilized to model inter-object dependencies and extract robust features. Contribution scores were computed using layer-wise relevance propagation and visualized via heatmaps and bar plots, to allow clinicians to trace the rationale behind the predictions. The results of an extensive evaluation demonstrated that this method yields high classification accuracy in distinguishing between autism spectrum disorder and typical development, and provides rule-aligned explanations that mirror clinical reasoning. The findings of this study provide a foundation for future real-world deployment of early screening, special education, and pediatric neuropsychological applications.
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 211138-211150
Open AccessInterpretable Autism Screening via Object-Based Analysis of Child-Drawn Sketches
Developmental assessment methods for children largely rely on subjective manual scoring, which leads to limitations in interpretability, consistency, and scalability. For high-stakes applications in clinical and educational environments, high accuracy and transparency are essential in decision-making, which underscores the growing importance of explainable artificial intelligence techniques for automated developmental assessment. To address these challenges, we propose an explainable deep learning framework for automated developmental assessment based on freehand portraits drawn by children. The proposed system integrates clinically evaluated criteria directly into the design of a computational model to ensure both accuracy and interpretability. Focusing specifically on unstructured freehand portrait sketches, the framework introduces an object-based contribution analysis that segments each sketch into semantically meaningful parts such as eyes, nose, ears, and torso, for localized contribution scoring. A customized vision transformer architecture equipped with relative position encoding and safe layer normalization was utilized to model inter-object dependencies and extract robust features. Contribution scores were computed using layer-wise relevance propagation and visualized via heatmaps and bar plots, to allow clinicians to trace the rationale behind the predictions. The results of an extensive evaluation demonstrated that this method yields high classification accuracy in distinguishing between autism spectrum disorder and typical development, and provides rule-aligned explanations that mirror clinical reasoning. The findings of this study provide a foundation for future real-world deployment of early screening, special education, and pediatric neuropsychological applications.
- 20 September 2025
- Published by Springer Nature
- p. 57-66
CARE-VL: A Domain-Specialized Vision-Language Model for Early ASD Screening
- 20 September 2025
- Published by Springer Nature
- p. 57-66
CARE-VL: A Domain-Specialized Vision-Language Model for Early ASD Screening
- Open Access
- 25 February 2025
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 147
Rule training by score-based supervised contrastive learning for sketch explanation
- 25 February 2025
- Published by Elsevier in Engineering Applications of Artificial Intelligence
- Vol. 147
Open AccessRule training by score-based supervised contrastive learning for sketch explanation
- Open Access
- 3 October 2024
- Published by Elsevier in Pattern Recognition Letters
- Vol. 186, 156-163
A unified framework to stereotyped behavior detection for screening Autism Spectrum Disorder
- 3 October 2024
- Published by Elsevier in Pattern Recognition Letters
- Vol. 186, 156-163
Open AccessA unified framework to stereotyped behavior detection for screening Autism Spectrum Disorder
- Open Access
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 137948-137956
Improving Domain Generalization in Appearance-Based Gaze Estimation With Consistency Regularization
Gaze estimation, a method for understanding human behavior by analyzing where a person is looking, has significant applications in various fields including advertising, driving assistance, medical diagnostics, and human-computer interaction. Although appearance-based methods have shown promising performance in uncontrolled environments, they often perform poorly when applied to similar but different domains due to variances in image quality, gaze distribution, and illumination. To overcome this limitation, this study aims to improve the domain generalization of appearance-based gaze estimation models using deep learning techniques. We propose an end-to-end deep learning approach that facilitates domain-agnostic feature learning and introduce a novel loss function, spherical gaze distance (SGD), and a regularization method, gaze consistency regularization (GCR). Our experiments, conducted using three commonly used datasets for appearance-based gaze estimation: ETH-XGaze, MPIIGaze, and GazeCapture, demonstrate the effectiveness of SGD and GCR. The results show that the proposed approach outperforms all the state-of-the-art methods on the domain generalization task and significantly improves performance when SGD and GCR are combined. These findings have important implications for the field of gaze estimation, suggesting that the proposed method could enhance the robustness and generalizability of gaze estimation models.
- 7 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 137948-137956
Open AccessImproving Domain Generalization in Appearance-Based Gaze Estimation With Consistency Regularization
Gaze estimation, a method for understanding human behavior by analyzing where a person is looking, has significant applications in various fields including advertising, driving assistance, medical diagnostics, and human-computer interaction. Although appearance-based methods have shown promising performance in uncontrolled environments, they often perform poorly when applied to similar but different domains due to variances in image quality, gaze distribution, and illumination. To overcome this limitation, this study aims to improve the domain generalization of appearance-based gaze estimation models using deep learning techniques. We propose an end-to-end deep learning approach that facilitates domain-agnostic feature learning and introduce a novel loss function, spherical gaze distance (SGD), and a regularization method, gaze consistency regularization (GCR). Our experiments, conducted using three commonly used datasets for appearance-based gaze estimation: ETH-XGaze, MPIIGaze, and GazeCapture, demonstrate the effectiveness of SGD and GCR. The results show that the proposed approach outperforms all the state-of-the-art methods on the domain generalization task and significantly improves performance when SGD and GCR are combined. These findings have important implications for the field of gaze estimation, suggesting that the proposed method could enhance the robustness and generalizability of gaze estimation models.
- Open Access
- 10 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 70865-70872
CNN Learning Strategy for Recognizing Facial Expressions
The ability to recognize facial expressions using computer vision is a crucial task, with numerous potential applications. Although deep neural networks have achieved high performance, it is still difficult to use them in the recognition of facial expressions. This is because different facial expressions have varying degrees of similarities among themselves, and numerous variations cause diversity in the same facial images. In this study, we propose a novel divide-and-conquer-based learning strategy to improve the performance of facial expression recognition (FER). The face area in an image was detected using MobileNet, and a ResNet-18 model was used as a backbone deep neural network for recognizing facial expressions. Thereafter, groups that included similar facial expressions were categorized by analyzing the confusion matrix, which is the inference result of the trained ResNet-18 model, and these similar facial expression groups were utilized to re-train the deep learning model. In the experiments, the proposed method was evaluated using two thermal (Tufts and RWTH) and two RGB (RAF and FER2013) datasets, and higher FER performances, which were 97.75% for Tufts, 86.11% for RWTH, 90.81% for RAF, and 77.83% for FER2013, were achieved. As such, the proposed method can accurately classify large amounts of facial expression data.
- 10 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 70865-70872
Open AccessCNN Learning Strategy for Recognizing Facial Expressions
The ability to recognize facial expressions using computer vision is a crucial task, with numerous potential applications. Although deep neural networks have achieved high performance, it is still difficult to use them in the recognition of facial expressions. This is because different facial expressions have varying degrees of similarities among themselves, and numerous variations cause diversity in the same facial images. In this study, we propose a novel divide-and-conquer-based learning strategy to improve the performance of facial expression recognition (FER). The face area in an image was detected using MobileNet, and a ResNet-18 model was used as a backbone deep neural network for recognizing facial expressions. Thereafter, groups that included similar facial expressions were categorized by analyzing the confusion matrix, which is the inference result of the trained ResNet-18 model, and these similar facial expression groups were utilized to re-train the deep learning model. In the experiments, the proposed method was evaluated using two thermal (Tufts and RWTH) and two RGB (RAF and FER2013) datasets, and higher FER performances, which were 97.75% for Tufts, 86.11% for RWTH, 90.81% for RAF, and 77.83% for FER2013, were achieved. As such, the proposed method can accurately classify large amounts of facial expression data.
- Open Access
- 6 January 2021
- Published by MDPI AG in Electronics
- Vol. 10 (2) , 99
Real-Time Hair Segmentation Using Mobile-Unet
We described a real-time hair segmentation method based on a fully convolutional network with the basic structure of an encoder–decoder. In one of the traditional computer vision techniques for hair segmentation, the mean shift and watershed methodologies suffer from inaccuracy and slow execution due to multi-step, complex image processing. It is also difficult to execute the process in real-time unless an optimization technique is applied to the partition. To solve this problem, we exploited Mobile-Unet using the U-Net segmentation model, which incorporates the optimization techniques of MobileNetV2. In experiments, hair segmentation accuracy was evaluated by different genders and races, and the average accuracy was 89.9%. By comparing the accuracy and execution speed of our model with those of other models in related studies, we confirmed that the proposed model achieved the same or better performance. As such, the results of hair segmentation can obtain hair information (style, color, length), which has a significant impact on human-robot interaction with people.
- 6 January 2021
- Published by MDPI AG in Electronics
- Vol. 10 (2) , 99
Open AccessReal-Time Hair Segmentation Using Mobile-Unet
We described a real-time hair segmentation method based on a fully convolutional network with the basic structure of an encoder–decoder. In one of the traditional computer vision techniques for hair segmentation, the mean shift and watershed methodologies suffer from inaccuracy and slow execution due to multi-step, complex image processing. It is also difficult to execute the process in real-time unless an optimization technique is applied to the partition. To solve this problem, we exploited Mobile-Unet using the U-Net segmentation model, which incorporates the optimization techniques of MobileNetV2. In experiments, hair segmentation accuracy was evaluated by different genders and races, and the average accuracy was 89.9%. By comparing the accuracy and execution speed of our model with those of other models in related studies, we confirmed that the proposed model achieved the same or better performance. As such, the results of hair segmentation can obtain hair information (style, color, length), which has a significant impact on human-robot interaction with people.
- Open Access
- 24 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 159172-159181
Toward Unbiased Facial Expression Recognition in the Wild via Cross-Dataset Adaptation
Despite various success in computer vision with facial images (e.g., face detection, recognition, and generation), facial expression recognition is still a challenging problem yet to be solved. This is because of simple but fundamental bottlenecks: 1) no global agreement on different facial expressions, 2) significant dataset biases that prevent cross-dataset analysis for a large-scale study, and 3) high class imbalance in in-the-wild datasets that causes inconsistency in predicting expressions in images using a machine learning algorithm. To tackle these issues, we propose a novel Deep Learning approach via adaptive cross-dataset scheme. We combine multiple in-the-wild datasets to secure sufficient training samples while minimizing dataset bias using ideas of reversal gradients to retain generality. For this, we introduce a flexible objective function that can control for skewed label distributions in the dataset. Incorporating these ideas, together with the ResNet pipeline as a backbone, we carried extensive experiments to validate our ideas using three independent in-the-wild facial expression datasets, which first confirmed bias from different datasets and yielded improved performance on facial expression recognition using the multi-site dataset.
- 24 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 159172-159181
Open AccessToward Unbiased Facial Expression Recognition in the Wild via Cross-Dataset Adaptation
Despite various success in computer vision with facial images (e.g., face detection, recognition, and generation), facial expression recognition is still a challenging problem yet to be solved. This is because of simple but fundamental bottlenecks: 1) no global agreement on different facial expressions, 2) significant dataset biases that prevent cross-dataset analysis for a large-scale study, and 3) high class imbalance in in-the-wild datasets that causes inconsistency in predicting expressions in images using a machine learning algorithm. To tackle these issues, we propose a novel Deep Learning approach via adaptive cross-dataset scheme. We combine multiple in-the-wild datasets to secure sufficient training samples while minimizing dataset bias using ideas of reversal gradients to retain generality. For this, we introduce a flexible objective function that can control for skewed label distributions in the dataset. Incorporating these ideas, together with the ResNet pipeline as a backbone, we carried extensive experiments to validate our ideas using three independent in-the-wild facial expression datasets, which first confirmed bias from different datasets and yielded improved performance on facial expression recognition using the multi-site dataset.
- Open Access
Face Recognition at a Distance for a Stand-Alone Access Control System
Although access control based on human face recognition has become popular in consumer applications, it still has several implementation issues before it can realize a stand-alone access control system. Owing to a lack of computational resources, lightweight and computationally efficient face recognition algorithms are required. The conventional access control systems require significant active cooperation from the users despite its non-aggressive nature. The lighting/illumination change is one of the most difficult and challenging problems for human-face-recognition-based access control applications. This paper presents the design and implementation of a user-friendly, stand-alone access control system based on human face recognition at a distance. The local binary pattern (LBP)-AdaBoost framework was employed for face and eyes detection, which is fast and invariant to illumination changes. It can detect faces and eyes of varied sizes at a distance. For fast face recognition with a high accuracy, the Gabor-LBP histogram framework was modified by substituting the Gabor wavelet with Gaussian derivative filters, which reduced the facial feature size by 40% of the Gabor-LBP-based facial features, and was robust to significant illumination changes and complicated backgrounds. The experiments on benchmark datasets produced face recognition accuracies of 97.27% on an E-face dataset and 99.06% on an XM2VTS dataset, respectively. The system achieved a 91.5% true acceptance rate with a 0.28% false acceptance rate and averaged a 5.26 frames/sec processing speed on a newly collected face image and video dataset in an indoor office environment.
Open AccessFace Recognition at a Distance for a Stand-Alone Access Control System
Although access control based on human face recognition has become popular in consumer applications, it still has several implementation issues before it can realize a stand-alone access control system. Owing to a lack of computational resources, lightweight and computationally efficient face recognition algorithms are required. The conventional access control systems require significant active cooperation from the users despite its non-aggressive nature. The lighting/illumination change is one of the most difficult and challenging problems for human-face-recognition-based access control applications. This paper presents the design and implementation of a user-friendly, stand-alone access control system based on human face recognition at a distance. The local binary pattern (LBP)-AdaBoost framework was employed for face and eyes detection, which is fast and invariant to illumination changes. It can detect faces and eyes of varied sizes at a distance. For fast face recognition with a high accuracy, the Gabor-LBP histogram framework was modified by substituting the Gabor wavelet with Gaussian derivative filters, which reduced the facial feature size by 40% of the Gabor-LBP-based facial features, and was robust to significant illumination changes and complicated backgrounds. The experiments on benchmark datasets produced face recognition accuracies of 97.27% on an E-face dataset and 99.06% on an XM2VTS dataset, respectively. The system achieved a 91.5% true acceptance rate with a 0.28% false acceptance rate and averaged a 5.26 frames/sec processing speed on a newly collected face image and video dataset in an indoor office environment.