JSJy-Yong Sohn

Jy-Yong Sohn

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jy-Yong Sohn has a total of 35 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Reference normalizations of Effort-To-Compress across length, alphabet size, and composition

    by Hyunwoo Jang, Jy-Yong Sohn, Zirui Huang

    Comparing finite symbolic complexity requires an explicit reference. Effort-To-Compress (ETC) counts the recursive pair substitutions needed to reduce a symbolic sequence to a constant. Its conventional normalization with L − 1 leaves systematic dependence on sequence length L and alphabet size k , confounding comparisons across datasets. We derive certified lower bounds for ETC and show that, for L ≥ k ≥ 2, the ceiling L − 1 is attainable if and only if L ≤ k 2 + k + 1. For fixed k ≥ 2, the global ensemble mean and maximum ETC grow as Θ k ( L/ log k L ) as L → ∞. These support two normalized ETC coordinates. Global-ensemble normalized ETC (nETC) maps a certified lower bound to zero and the uniform ensemble mean at the same L and k to one. Composition-conditioned normalized ETC (cETC) uses corresponding references restricted to the observed symbol composition. In Potts chains, nETC substantially reduced dependence on length and alphabet size. cETC further reduced differences associated with unequal symbol frequencies, while retaining sensitivity to temporal dependence. In human chromosome 22, the exploratory predictor nETC – cETC outperformed LZ76 in distinguishing CpG islands and long interspersed nuclear elements from genomic background with AUROC of 0.947 and 0.848. Matched references therefore enable interpretable comparisons of symbolic complexity across sequence lengths, alphabet sizes, and compositions.

  • Open Access

    FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation

    by Chanyeol Choi, Jihoon Kwon, Jaeseon Ha, Hojun Choi, Chaewoon Kim, Yongjae Lee, Jy-Yong Sohn, Alejandro Lopez-Lira


  • Open Access

    Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study

    by Jae-Hee So, Joonhwan Chang, Eunji Kim, Junho Na, JiYeon Choi, Jy-Yong Sohn, Byung-Hoon Kim, Sang Hui Chu

    Background: Recent advancements in large language models (LLMs) have accelerated their use across various domains. Psychiatric interviews, which are goal-oriented and structured, represent a significantly underexplored area where LLMs can provide substantial value. In this study, we explore the application of LLMs to enhance psychiatric interviews by analyzing counseling data from North Korean defectors who have experienced traumatic events and mental health issues. Objective: This study aims to investigate whether LLMs can (1) delineate parts of the conversation that suggest psychiatric symptoms and identify those symptoms, and (2) summarize stressors and symptoms based on the interview dialogue transcript. Methods: Given the interview transcripts, we align the LLMs to perform 3 tasks: (1) extracting stressors from the transcripts, (2) delineating symptoms and their indicative sections, and (3) summarizing the patients based on the extracted stressors and symptoms. These 3 tasks address the 2 objectives, where delineating symptoms is based on the output from the second task, and generating the summary of the interview incorporates the outputs from all 3 tasks. In this context, the transcript data were labeled by mental health experts for the training and evaluation of the LLMs. Results: First, we present the performance of LLMs in estimating (1) the transcript sections related to psychiatric symptoms and (2) the names of the corresponding symptoms. In the zero-shot inference setting using the GPT-4 Turbo model, 73 out of 102 transcript segments demonstrated a recall mid-token distance d<20 for estimating the sections associated with the symptoms. For evaluating the names of the corresponding symptoms, the fine-tuning method demonstrates a performance advantage over the zero-shot inference setting of the GPT-4 Turbo model. On average, the fine-tuning method achieves an accuracy of 0.82, a precision of 0.83, a recall of 0.82, and an F1-score of 0.82. Second, the transcripts are used to generate summaries for each interviewee using LLMs. This generative task was evaluated using metrics such as Generative Evaluation (G-Eval) and Bidirectional Encoder Representations from Transformers Score (BERTScore). The summaries generated by the GPT-4 Turbo model, utilizing both symptom and stressor information, achieve high average G-Eval scores: coherence of 4.66, consistency of 4.73, fluency of 2.16, and relevance of 4.67. Furthermore, it is noted that the use of retrieval-augmented generation did not lead to a significant improvement in performance. Conclusions: LLMs, using either (1) appropriate prompting techniques or (2) fine-tuning methods with data labeled by mental health experts, achieved an accuracy of over 0.8 for the symptom delineation task when measured across all segments in the transcript. Additionally, they attained a G-Eval score of over 4.6 for coherence in the summarization task. This research contributes to the emerging field of applying LLMs in psychiatric interviews and demonstrates their potential effectiveness in assisting mental health practitioners.


  • Open Access

    Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study (Preprint)

    by Jae-Hee So, Joonhwan Chang, Eunji Kim, Junho Na, JiYeon Choi, Jy-Yong Sohn, Byung-Hoon Kim, Sang Hui Chu

    BACKGROUND Recent advancements in large language models (LLMs) have accelerated their use across various domains. Psychiatric interviews, which are goal-oriented and structured, represent a significantly underexplored area where LLMs can provide substantial value. In this study, we explore the application of LLMs to enhance psychiatric interviews by analyzing counseling data from North Korean defectors who have experienced traumatic events and mental health issues. OBJECTIVE This study aims to investigate whether LLMs can (1) delineate parts of the conversation that suggest psychiatric symptoms and identify those symptoms, and (2) summarize stressors and symptoms based on the interview dialogue transcript. METHODS Given the interview transcripts, we align the LLMs to perform 3 tasks: (1) extracting stressors from the transcripts, (2) delineating symptoms and their indicative sections, and (3) summarizing the patients based on the extracted stressors and symptoms. These 3 tasks address the 2 objectives, where delineating symptoms is based on the output from the second task, and generating the summary of the interview incorporates the outputs from all 3 tasks. In this context, the transcript data were labeled by mental health experts for the training and evaluation of the LLMs. RESULTS First, we present the performance of LLMs in estimating (1) the transcript sections related to psychiatric symptoms and (2) the names of the corresponding symptoms. In the zero-shot inference setting using the GPT-4 Turbo model, 73 out of 102 transcript segments demonstrated a recall mid-token distance d<20 for estimating the sections associated with the symptoms. For evaluating the names of the corresponding symptoms, the fine-tuning method demonstrates a performance advantage over the zero-shot inference setting of the GPT-4 Turbo model. On average, the fine-tuning method achieves an accuracy of 0.82, a precision of 0.83, a recall of 0.82, and an F1-score of 0.82. Second, the transcripts are used to generate summaries for each interviewee using LLMs. This generative task was evaluated using metrics such as Generative Evaluation (G-Eval) and Bidirectional Encoder Representations from Transformers Score (BERTScore). The summaries generated by the GPT-4 Turbo model, utilizing both symptom and stressor information, achieve high average G-Eval scores: coherence of 4.66, consistency of 4.73, fluency of 2.16, and relevance of 4.67. Furthermore, it is noted that the use of retrieval-augmented generation did not lead to a significant improvement in performance. CONCLUSIONS LLMs, using either (1) appropriate prompting techniques or (2) fine-tuning methods with data labeled by mental health experts, achieved an accuracy of over 0.8 for the symptom delineation task when measured across all segments in the transcript. Additionally, they attained a G-Eval score of over 4.6 for coherence in the summarization task. This research contributes to the emerging field of applying LLMs in psychiatric interviews and demonstrates their potential effectiveness in assisting mental health practitioners.


  • Open Access

    Improving Multi-lingual Alignment Through Soft Contrastive Learning

    by Minsu Park, Seyeon Choi, Chanyeol Choi, Jun-Seong Kim, Jy-Yong Sohn

    Minsu Park, Seyeon Choi, Chanyeol Choi, Jun-Seong Kim, Jy-yong Sohn. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop). 2024.


  • Open Access

    Can We Find Strong Lottery Tickets in Generative Models?

    by Sangyeop Yeo, Yoojin Jang, Jy-Yong Sohn, Dongyoon Han, Jaejun Yoo

    Yes. In this paper, we investigate strong lottery tickets in generative models, the subnetworks that achieve good generative performance without any weight update. Neural network pruning is considered the main cornerstone of model compression for reducing the costs of computation and memory. Unfortunately, pruning a generative model has not been extensively explored, and all existing pruning algorithms suffer from excessive weight-training costs, performance degradation, limited generalizability, or complicated training. To address these problems, we propose to find a strong lottery ticket via moment-matching scores. Our experimental results show that the discovered subnetwork can perform similarly or better than the trained dense model even when only 10% of the weights remain. To the best of our knowledge, we are the first to show the existence of strong lottery tickets in generative models and provide an algorithm to find it stably. Our code and supplementary materials are publicly available at https://lait-cvlab.github.io/SLT-in-Generative-Models/.


  • Open Access

    Can We Find Strong Lottery Tickets in Generative Models?

    by Sangyeop Yeo, Yoojin Jang, Jy-Yong Sohn, Dongyoon Han, Jaejun Yoo

    Yes. In this paper, we investigate strong lottery tickets in generative models, the subnetworks that achieve good generative performance without any weight update. Neural network pruning is considered the main cornerstone of model compression for reducing the costs of computation and memory. Unfortunately, pruning a generative model has not been extensively explored, and all existing pruning algorithms suffer from excessive weight-training costs, performance degradation, limited generalizability, or complicated training. To address these problems, we propose to find a strong lottery ticket via moment-matching scores. Our experimental results show that the discovered subnetwork can perform similarly or better than the trained dense model even when only 10% of the weights remain. To the best of our knowledge, we are the first to show the existence of strong lottery tickets in generative models and provide an algorithm to find it stably. Our code and supplementary materials are publicly available.

  • Coded Wireless Distributed Computing With Packet Losses and Retransmissions

    by Dong-Jun Han, Jy-Yong Sohn, Jaekyun Moon

    In wireless distributed computing systems, mobile devices that are connected wirelessly to the Fog (e.g., small base stations) collaboratively solve a given computational task. Unfortunately, wireless distributed computing systems suffer from packet losses due to severe channel fading. Moreover, a wireless device can drop out of the system when leaving the coverage of a master node in the Fog layer. We model this unreliability between a device and a master node as a packet erasure channel. When a packet fails to be detected at the receiver, the corresponding packet is retransmitted, which would significantly increase the overall run-time to finish the task. We take a coding-theoretic approach to tackle this straggler-like problem in wireless distributed computing. We first investigate the expected latency using an $(n, k)$ maximum-distance separable (MDS) code. We obtain the lower and upper bounds on the latency in closed forms and provide guidelines to design MDS codes depending on the channel condition characterized by packet erasure probability. Then, we introduce another important performance metric called minimum latency , and also provide guidelines on designing optimal codes. Based on optimal codes, we obtain the performance curves of achievable minimum latency and achievable workload as functions of packet erasure probability.


  • Hierarchical Broadcast Coding: Expediting Distributed Learning at the Wireless Edge

    by Dong-Jun Han, Jy-Yong Sohn, Jaekyun Moon

    Distributed learning plays a key role in reducing the training time of modern deep neural networks with massive datasets. In this paper, we consider a distributed learning problem where gradient computation is carried out over a number of computing devices at the wireless edge. We propose hierarchical broadcast coding, a provable coding-theoretic framework to speed up distributed learning at the wireless edge. Our contributions are threefold. First, motivated by the hierarchical nature of real-world edge computing systems, we propose a layered code which mitigates the effects of not only packet losses at the wireless computing nodes but also straggling access points (APs or small base stations). Second, by strategically allocating data partitions to nodes in the overlapping areas between cells, our technique achieves the fundamental lower bound on computational load to combat stragglers. Finally, we take advantage of the broadcast nature of wireless networks by which wireless devices in the overlapping cell coverage broadcast to more than one AP. This further reduces the overall training time in the presence of straggling APs. Experimental results on Amazon EC2 confirm the advantage of the proposed methods in speeding up learning. Our design targets any gradient descent based learning algorithms, including linear/logistic regressions and deep learning.


  • Open Access

    Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks

    by Jy-Yong Sohn, Dong-Jun Han, Beongjun Choi, Jaekyun Moon

    Recent advances in large-scale distributed learning algorithms have enabled communication-efficient training via SignSGD. Unfortunately, a major issue continues to plague distributed learning: namely, Byzantine failures may incur serious degradation in learning accuracy. This paper proposes Election Coding, a coding-theoretic framework to guarantee Byzantine-robustness for SignSGD with Majority Vote, which uses minimum worker-master communication in both directions. The suggested framework explores new information-theoretic limits of finding the majority opinion when some workers could be malicious, and paves the road to implement robust and efficient distributed learning algorithms. Under this framework, we construct two types of explicit codes, random Bernoulli codes and deterministic algebraic codes, that can tolerate Byzantine attacks with a controlled amount of computational redundancy. For the Bernoulli codes, we provide upper bounds on the error probability in estimating the majority opinion, which give useful insights into code design for tolerating Byzantine attacks. As for deterministic codes, we construct an explicit code which perfectly tolerates Byzantines, and provide tight upper/lower bounds on the minimum required computational redundancy. Finally, the Byzantine-tolerance of the suggested coding schemes is confirmed by deep learning experiments on Amazon EC2 using Python with MPI4py package.