Hyunil Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Hyunil Kim has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Changho Seo
Kongju National University
2 shared publications - Haehyun Cho
Soongsil University
2 shared publications - Minyeong Choe
Chosun University
2 shared publications Changho Seo
Department of Convergence Science, Kongju National University, Chungcheongnam-do, Gongju-si, 32588, South Korea
1 shared publicationChanghoon Lim
Department of Artificial Intelligence and Software Engineering, Chosun University, Gwangju, 61467, Republic of Korea
1 shared publicationDeepak Upreti
Department of Convergence Science, Kongju National University, Chungcheongnam-do, Gongju-si, 32588, South Korea
1 shared publication
Latest Publications
- Open Access
Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms
Understanding how Transformer-based language models store and retrieve factual associations is critical for improving interpretability and enabling targeted model editing. Prior work, primarily on GPT-style models, has identified MLP modules in early layers as key contributors to factual recall. However, it remains unclear whether these findings generalize across different autoregressive architectures. To address this, we conduct a comprehensive evaluation of factual recall across several models -- including GPT, LLaMA, Qwen, and DeepSeek -- analyzing where and how factual information is encoded and accessed. Consequently, we find that Qwen-based models behave differently from previous patterns: attention modules in the earliest layers contribute more to factual recall than MLP modules. Our findings suggest that even within the autoregressive Transformer family, architectural variations can lead to fundamentally different mechanisms of factual recall.
Open AccessDo All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms
Understanding how Transformer-based language models store and retrieve factual associations is critical for improving interpretability and enabling targeted model editing. Prior work, primarily on GPT-style models, has identified MLP modules in early layers as key contributors to factual recall. However, it remains unclear whether these findings generalize across different autoregressive architectures. To address this, we conduct a comprehensive evaluation of factual recall across several models -- including GPT, LLaMA, Qwen, and DeepSeek -- analyzing where and how factual information is encoded and accessed. Consequently, we find that Qwen-based models behave differently from previous patterns: attention modules in the earliest layers contribute more to factual recall than MLP modules. Our findings suggest that even within the autoregressive Transformer family, architectural variations can lead to fundamentally different mechanisms of factual recall.
- Open Access
- 1 January 2025
- Published by Tech Science Press in Computer Modeling in Engineering & Sciences
- Vol. 145 (2) , 2617-2630
How Robust Are Language Models against Backdoors in Federated Learning?
Federated Learning enables privacy-preserving training of Transformer-based language models, but remains vulnerable to backdoor attacks that compromise model reliability. This paper presents a comparative analysis of defense strategies against both classical and advanced backdoor attacks, evaluated across autoencoding and autoregressive models. Unlike prior studies, this work provides the first systematic comparison of perturbation-based, screening-based, and hybrid defenses in Transformer-based FL environments. Our results show that screening-based defenses consistently outperform perturbation-based ones, effectively neutralizing most attacks across architectures. However, this robustness comes with significant computational overhead, revealing a clear trade-off between security and efficiency. By explicitly identifying this trade-off, our study advances the understanding of defense strategies in federated learning and highlights the need for lightweight yet effective screening methods for trustworthy deployment in diverse application domains.
- 1 January 2025
- Published by Tech Science Press in Computer Modeling in Engineering & Sciences
- Vol. 145 (2) , 2617-2630
Open AccessHow Robust Are Language Models against Backdoors in Federated Learning?
Federated Learning enables privacy-preserving training of Transformer-based language models, but remains vulnerable to backdoor attacks that compromise model reliability. This paper presents a comparative analysis of defense strategies against both classical and advanced backdoor attacks, evaluated across autoencoding and autoregressive models. Unlike prior studies, this work provides the first systematic comparison of perturbation-based, screening-based, and hybrid defenses in Transformer-based FL environments. Our results show that screening-based defenses consistently outperform perturbation-based ones, effectively neutralizing most attacks across architectures. However, this robustness comes with significant computational overhead, revealing a clear trade-off between security and efficiency. By explicitly identifying this trade-off, our study advances the understanding of defense strategies in federated learning and highlights the need for lightweight yet effective screening methods for trustworthy deployment in diverse application domains.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 28482-28501
Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms
Minyeong Choe, Haehyun Cho, Changho Seo, Hyunil Kim. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 28482-28501
Open AccessDo All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms
Minyeong Choe, Haehyun Cho, Changho Seo, Hyunil Kim. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.
- Open Access
- 1 January 2024
- Published by Tech Science Press in Computer Modeling in Engineering & Sciences
- Vol. 140 (3) , 2239-2274
A Comprehensive Survey on Federated Learning in the Healthcare Area: Concept and Applications
Federated learning is an innovative machine learning technique that deals with centralized data storage issues while maintaining privacy and security. It involves constructing machine learning models using datasets spread across several data centers, including medical facilities, clinical research facilities, Internet of Things devices, and even mobile devices. The main goal of federated learning is to improve robust models that benefit from the collective knowledge of these disparate datasets without centralizing sensitive information, reducing the risk of data loss, privacy breaches, or data exposure. The application of federated learning in the healthcare industry holds significant promise due to the wealth of data generated from various sources, such as patient records, medical imaging, wearable devices, and clinical research surveys. This research conducts a systematic evaluation and highlights essential issues for the selection and implementation of federated learning approaches in healthcare. It evaluates the effectiveness of federated learning strategies in the field of healthcare. It offers a systematic analysis of federated learning in the healthcare domain, encompassing the evaluation metrics employed. In addition, this study highlights the increasing interest in federated learning applications in healthcare among scholars and provides foundations for further studies.
- 1 January 2024
- Published by Tech Science Press in Computer Modeling in Engineering & Sciences
- Vol. 140 (3) , 2239-2274
Open AccessA Comprehensive Survey on Federated Learning in the Healthcare Area: Concept and Applications
Federated learning is an innovative machine learning technique that deals with centralized data storage issues while maintaining privacy and security. It involves constructing machine learning models using datasets spread across several data centers, including medical facilities, clinical research facilities, Internet of Things devices, and even mobile devices. The main goal of federated learning is to improve robust models that benefit from the collective knowledge of these disparate datasets without centralizing sensitive information, reducing the risk of data loss, privacy breaches, or data exposure. The application of federated learning in the healthcare industry holds significant promise due to the wealth of data generated from various sources, such as patient records, medical imaging, wearable devices, and clinical research surveys. This research conducts a systematic evaluation and highlights essential issues for the selection and implementation of federated learning approaches in healthcare. It evaluates the effectiveness of federated learning strategies in the field of healthcare. It offers a systematic analysis of federated learning in the healthcare domain, encompassing the evaluation metrics employed. In addition, this study highlights the increasing interest in federated learning applications in healthcare among scholars and provides foundations for further studies.
- Open Access
An Overview of Recent Advances in Greenhouse Strawberry Cultivation Using Deep Learning Techniques: A Review for Strawberry Practitioners
Strawberry (Fragaria × ananassa Duch.) has been widely accepted as the “Queen of Fruits”. It has been identified as having high levels of vitamin C and antioxidants that are beneficial for maintaining cardiovascular health and maintaining blood sugar levels. The implementation of advanced techniques like precision agriculture (PA) is crucial for enhancing production compared to conventional farming methods. In recent years, the successful application of deep learning models was represented by convolutional neural networks (CNNs) in a variety of disciplines of computer vision (CV). Due to the dearth of a comprehensive and detailed discussion on the application of deep learning to strawberry cultivation, a particular review of recent technologies is needed. This paper provides an overview of recent advancements in strawberry cultivation utilizing Deep Learning (DL) techniques. It provides a comprehensive understanding of the most up-to-date techniques and methodologies used in this field by examining recent research. It also discusses the recent advanced variants of the DL model, along with a fundamental overview of CNN architecture. In addition, techniques for fine-tuning DL models have been covered. Besides, various strawberry-planting-related datasets were examined in the literature, and the limitations of using research models for real-time research have been discussed.
Open AccessAn Overview of Recent Advances in Greenhouse Strawberry Cultivation Using Deep Learning Techniques: A Review for Strawberry Practitioners
Strawberry (Fragaria × ananassa Duch.) has been widely accepted as the “Queen of Fruits”. It has been identified as having high levels of vitamin C and antioxidants that are beneficial for maintaining cardiovascular health and maintaining blood sugar levels. The implementation of advanced techniques like precision agriculture (PA) is crucial for enhancing production compared to conventional farming methods. In recent years, the successful application of deep learning models was represented by convolutional neural networks (CNNs) in a variety of disciplines of computer vision (CV). Due to the dearth of a comprehensive and detailed discussion on the application of deep learning to strawberry cultivation, a particular review of recent technologies is needed. This paper provides an overview of recent advancements in strawberry cultivation utilizing Deep Learning (DL) techniques. It provides a comprehensive understanding of the most up-to-date techniques and methodologies used in this field by examining recent research. It also discusses the recent advanced variants of the DL model, along with a fundamental overview of CNN architecture. In addition, techniques for fine-tuning DL models have been covered. Besides, various strawberry-planting-related datasets were examined in the literature, and the limitations of using research models for real-time research have been discussed.