Taeseok Yang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Taeseok Yang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Chungsoo Kim
Department of Biomedical Sciences, Graduate School of Ajou University, 164 Worldcup-ro, Suwon, 16499, Republic of Korea
1 shared publication - Jaehyeong Cho
Department of Research, Keimyung University Dongsan Medical Center, 1035 Dalgubeol-daero, Daegu, 42601, Republic of Korea
1 shared publication - Junhyuk Chang
Department of Biomedical Sciences, Graduate School of Ajou University, 164 Worldcup-ro, Suwon, 16499, Republic of Korea
1 shared publication - Kyungseon Choi
Section of Cardiovascular Medicine, Yale School of Medicine, 195 Church Street, New Haven, CT, 06510, USA
1 shared publication - Minkook Son
Department of Physiology, Dong-A University College of Medicine, Busan, Korea
1 shared publication - Yeonghyeon Kim
Interdisciplinary Program in Bioengineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea
1 shared publication
Latest Publications
- Open Access
MENTOR: An Evidence-Informed Large Language Model-Based Multi-Agent System for Efficient Selection of Institution-Specific Negative Control Outcomes
Objective: Negative control outcomes (NCOs) help detect and correct systematic error in observational studies, yet their selection remains largely manual. The Common Evidence Model (CEM) can generate candidate lists, but researchers must still screen them for plausibility as true nulls and for sufficient observation in each institution’s data. We developed and evaluated MENTOR (Multi-agent Evidence-informed Negative conTrol Outcome selection with Reasoning), a multi-agent system of large language models (LLMs) that combines CEM evidence with LLM reasoning to select institution-specific NCOs. Methods: Four LLM agents operate on an institution’s Observational Medical Outcomes Partnership common data model. A Curator summarizes drug labels, a Suggester assesses locally frequent candidates with CEM evidence and a first-pass LLM judgment, an Adjudicator resolves their disagreements, and a Diagnostician relaxes the patient-count threshold until a target number of NCOs is met. Using Qwen3.6-27B, we evaluated 349 candidates for three drug pairs in a Korean tertiary hospital’s electronic health records (2.9 million patients) against a blinded LLM judge from a different model family, validated by three experts on 90 candidates. We compared accuracy, positive predictive value (PPV), and sensitivity with CEM-only and first-pass-LLM-only selection. Results: MENTOR proposed 243 NCOs versus 121 by CEM-only, with higher accuracy (0.83 [0.79–0.87] vs. 0.53 [0.48–0.58]), sensitivity (0.89 vs. 0.41), and PPV (0.86 vs. 0.79). Judge–expert agreement was κ=0.63. Adjudication, needed for 44.1% of candidates, added little accuracy, and only with reasoning mode. Concordant acceptances, which bypass it, contained about half of the false positives. Only 7%–9% of CEM-proposed outcomes had at least 10,000 patients at the institution. Conclusion: Combining LLM judgments with CEM evidence and local data yielded more NCOs, with higher PPV, than CEM alone. Restricting reasoning to discordant candidates kept computation time below 40% of that with reasoning in all agents. Every decision comes with its evidence and rationale for investigators’ final review.
Open AccessMENTOR: An Evidence-Informed Large Language Model-Based Multi-Agent System for Efficient Selection of Institution-Specific Negative Control Outcomes
Objective: Negative control outcomes (NCOs) help detect and correct systematic error in observational studies, yet their selection remains largely manual. The Common Evidence Model (CEM) can generate candidate lists, but researchers must still screen them for plausibility as true nulls and for sufficient observation in each institution’s data. We developed and evaluated MENTOR (Multi-agent Evidence-informed Negative conTrol Outcome selection with Reasoning), a multi-agent system of large language models (LLMs) that combines CEM evidence with LLM reasoning to select institution-specific NCOs. Methods: Four LLM agents operate on an institution’s Observational Medical Outcomes Partnership common data model. A Curator summarizes drug labels, a Suggester assesses locally frequent candidates with CEM evidence and a first-pass LLM judgment, an Adjudicator resolves their disagreements, and a Diagnostician relaxes the patient-count threshold until a target number of NCOs is met. Using Qwen3.6-27B, we evaluated 349 candidates for three drug pairs in a Korean tertiary hospital’s electronic health records (2.9 million patients) against a blinded LLM judge from a different model family, validated by three experts on 90 candidates. We compared accuracy, positive predictive value (PPV), and sensitivity with CEM-only and first-pass-LLM-only selection. Results: MENTOR proposed 243 NCOs versus 121 by CEM-only, with higher accuracy (0.83 [0.79–0.87] vs. 0.53 [0.48–0.58]), sensitivity (0.89 vs. 0.41), and PPV (0.86 vs. 0.79). Judge–expert agreement was κ=0.63. Adjudication, needed for 44.1% of candidates, added little accuracy, and only with reasoning mode. Concordant acceptances, which bypass it, contained about half of the false positives. Only 7%–9% of CEM-proposed outcomes had at least 10,000 patients at the institution. Conclusion: Combining LLM judgments with CEM evidence and local data yielded more NCOs, with higher PPV, than CEM alone. Restricting reasoning to discordant candidates kept computation time below 40% of that with reasoning in all agents. Every decision comes with its evidence and rationale for investigators’ final review.