Shih-Yi Lin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Shih-Yi Lin has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Chia-Hung Kao
China Medical University Hospital
1 shared publication - Chien-Sheng Hsu
Department of Neurology, China Medical University Hospital, Taichung 404, Taiwan
1 shared publication - Pei-Chun Yeh
Artificial Intelligence and Robotics Innovation Center, China Medical University Hospital, Taichung 404, Taiwan
1 shared publication - Shih-Sheng Chang
School of Medicine, China Medical University, Taichung, Taiwan
1 shared publication - Wu-Huei Hsu
Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, China Medical University Hospital, Taichung, Taiwan
1 shared publication - Ying-Yu Hsu
National Changhua Senior High School, Changhua 500, Taiwan
1 shared publication
Latest Publications
- Open Access
- 5 March 2026
- Published by MDPI AG in Bioengineering
- Vol. 13 (3) , 302
Benchmarking Large Language Models on the Taiwan Neurology Board Examinations (2018–2024): A Comparative Evaluation of GPT-4o, GPT-o1, DeepSeek-V3, and DeepSeek-R1
Background and Purpose: Neurology requires integration of clinical reasoning, imaging interpretation, and current knowledge, making it an ideal field for evaluating large language models (LLMs). Methods: Using 1715 questions from the Taiwan Neurology Board Examination (2018–2024), we assessed four LLMs—GPT-4o, GPT-o1, DeepSeek-V3, and DeepSeek-R1—across four formats: single-choice, multiple-choice, true–false, and image-based items. Results: GPT-o1 achieved the highest overall accuracy (83.86%) and demonstrated strong performance on cognitively demanding tasks (82.50% on true–false; 77.26% on image-based). DeepSeek-V3 scored lowest (65.62%) and showed the greatest variability. Statistical analyses confirmed significant inter-model differences (p < 0.01). Accuracy declined across all models in 2024, coinciding with shifts in question design. DeepSeek-R1 was further penalized by alignment-based refusals, resulting in up to 3.81% score loss. Conclusions: These results position the Taiwan Neurology Board Exam as a rigorous benchmark for LLM evaluation and underscore GPT-o1’s potential utility in neurology education and decision support.
- 5 March 2026
- Published by MDPI AG in Bioengineering
- Vol. 13 (3) , 302
Open AccessBenchmarking Large Language Models on the Taiwan Neurology Board Examinations (2018–2024): A Comparative Evaluation of GPT-4o, GPT-o1, DeepSeek-V3, and DeepSeek-R1
Background and Purpose: Neurology requires integration of clinical reasoning, imaging interpretation, and current knowledge, making it an ideal field for evaluating large language models (LLMs). Methods: Using 1715 questions from the Taiwan Neurology Board Examination (2018–2024), we assessed four LLMs—GPT-4o, GPT-o1, DeepSeek-V3, and DeepSeek-R1—across four formats: single-choice, multiple-choice, true–false, and image-based items. Results: GPT-o1 achieved the highest overall accuracy (83.86%) and demonstrated strong performance on cognitively demanding tasks (82.50% on true–false; 77.26% on image-based). DeepSeek-V3 scored lowest (65.62%) and showed the greatest variability. Statistical analyses confirmed significant inter-model differences (p < 0.01). Accuracy declined across all models in 2024, coinciding with shifts in question design. DeepSeek-R1 was further penalized by alignment-based refusals, resulting in up to 3.81% score loss. Conclusions: These results position the Taiwan Neurology Board Exam as a rigorous benchmark for LLM evaluation and underscore GPT-o1’s potential utility in neurology education and decision support.