HKHoon Kim

Hoon Kim

*This is an algorithmically generated author record based on publicly available information.

Hoon Kim has a total of 5 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Optimizing Small Local Language Models for Culturally Competent Mental Health Counseling: Comparative Evaluation With GPT-4o by Psychiatrists

    by Jaewook Han, Beomgi So, Whanbo Jung, Minwoo Kim, Hoon Kim, Daun Shin

    Background: Large language models (LLMs) show promise for mental health counseling applications. However, their deployment faces significant challenges, including data privacy concerns requiring on-premise solutions, high operational costs, and Western-centric biases that limit cultural appropriateness for non–English-speaking populations. Objective: This study aimed to develop and evaluate culturally competent small language models (SLMs) with approximately 7 to 8 billion parameters for Korean mental health counseling and to validate a 2-stage hybrid evaluation framework combining LLM-as-a-Judge screening with expert clinical validation. Methods: We constructed a dataset of 1558 diary-based counseling interactions from a Korean mental health mobile app (Maeumpyeonuijeom). Three Korean-optimized base models (SKT A.X 4.0 Light {7B}, LG EXAONE 3.5 {7.8B}, and Meta Llama 3.1 {8B}) were fine-tuned using an aggressive fine-tuning strategy with high-rank low-rank adaptation (r=256) over 20 epochs. Inference parameters were optimized using GPT-5 (gpt-5-2025-08-07) as an LLM-as-a-Judge. The optimized SLMs and GPT-4o (gpt-4o-2024-08-06) were evaluated by 10 board-certified psychiatrists (mean clinical experience of approximately 10 years) on 50 counseling scenarios across 4 dimensions (individualization, supportiveness, cultural appropriateness, and safety) using 5-point Likert scales. Wilcoxon signed-rank tests with Holm correction were used for statistical comparison, and intraclass correlation coefficients (ICCs) assessed interrater reliability. Results: No statistically significant difference from GPT-4o was detected for SKT A.X 4.0 Light in supportiveness (mean 3.39, SD 0.74 vs mean 3.45, SD 0.90; P=.26) and cultural appropriateness (mean 3.30, SD 0.77 vs mean 3.29, SD 0.79; P=.58). SKT A.X 4.0 Light ranked first in cultural appropriateness among all models. Interrater reliability analysis revealed higher agreement among psychiatrists for local SLMs (ICC=0.575 for SKT A.X 4.0 Light in cultural appropriateness) compared with GPT-4o (ICC=0.211). GPT-4o maintained significantly higher safety scores than all SLMs (P<.001). The LLM-as-a-Judge demonstrated moderate-to-good correlation with human evaluation but exhibited a ceiling effect in safety scoring. Conclusions: A 7 to 8-billion-parameter SLM fine-tuned on culturally specific data can achieve performance comparable to GPT-4o in supportiveness and cultural appropriateness for mental health counseling. These findings support the feasibility of locally deployable, privacy-preserving mental health AI systems based on culturally adapted SLMs. However, the safety gap necessitates human expert oversight, and success depends on the availability of high-quality language-specific base models.

  • Open Access

    Optimizing Small Local Language Models for Culturally Competent Mental Health Counseling: Comparative Evaluation With GPT-4o by Psychiatrists (Preprint)

    by Jaewook Han, Beomgi So, Whanbo Jung, Minwoo Kim, Hoon Kim, Daun Shin

    BACKGROUND Large language models (LLMs) show promise for mental health counseling applications. However, their deployment faces significant challenges, including data privacy concerns requiring on-premise solutions, high operational costs, and Western-centric biases that limit cultural appropriateness for non–English-speaking populations. OBJECTIVE This study aimed to develop and evaluate culturally competent small language models (SLMs) with approximately 7 to 8 billion parameters for Korean mental health counseling and to validate a 2-stage hybrid evaluation framework combining LLM-as-a-Judge screening with expert clinical validation. METHODS We constructed a dataset of 1558 diary-based counseling interactions from a Korean mental health mobile app (Maeumpyeonuijeom). Three Korean-optimized base models (SKT A.X 4.0 Light {7B}, LG EXAONE 3.5 {7.8B}, and Meta Llama 3.1 {8B}) were fine-tuned using an aggressive fine-tuning strategy with high-rank low-rank adaptation (r=256) over 20 epochs. Inference parameters were optimized using GPT-5 (gpt-5-2025-08-07) as an LLM-as-a-Judge. The optimized SLMs and GPT-4o (gpt-4o-2024-08-06) were evaluated by 10 board-certified psychiatrists (mean clinical experience of approximately 10 years) on 50 counseling scenarios across 4 dimensions (individualization, supportiveness, cultural appropriateness, and safety) using 5-point Likert scales. Wilcoxon signed-rank tests with Holm correction were used for statistical comparison, and intraclass correlation coefficients (ICCs) assessed interrater reliability. RESULTS No statistically significant difference from GPT-4o was detected for SKT A.X 4.0 Light in supportiveness (mean 3.39, SD 0.74 vs mean 3.45, SD 0.90; P=.26) and cultural appropriateness (mean 3.30, SD 0.77 vs mean 3.29, SD 0.79; P=.58). SKT A.X 4.0 Light ranked first in cultural appropriateness among all models. Interrater reliability analysis revealed higher agreement among psychiatrists for local SLMs (ICC=0.575 for SKT A.X 4.0 Light in cultural appropriateness) compared with GPT-4o (ICC=0.211). GPT-4o maintained significantly higher safety scores than all SLMs (P<.001). The LLM-as-a-Judge demonstrated moderate-to-good correlation with human evaluation but exhibited a ceiling effect in safety scoring. CONCLUSIONS A 7 to 8-billion-parameter SLM fine-tuned on culturally specific data can achieve performance comparable to GPT-4o in supportiveness and cultural appropriateness for mental health counseling. These findings support the feasibility of locally deployable, privacy-preserving mental health AI systems based on culturally adapted SLMs. However, the safety gap necessitates human expert oversight, and success depends on the availability of high-quality language-specific base models.