Seok Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Seok Kim has a total of 91 co-authors. Here are the top co-authors by number of shared publications:
- Sooyoung Yoo
Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
28 shared publications - Hee Hwang
Department of Pediatrics, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
8 shared publications - Ho-Young Lee
Department of Nuclear Medicine, Seoul National University Bundang Hospital, Seongnam, Korea
8 shared publications - Leonard Sunwoo
Seoul National University Bundang Hospital
6 shared publications - Borham Kim
Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
5 shared publications - Hyunyoung Baek
Office of eHealth Research and Business, Seoul National University Bundang Hospital, Seongnam-si, Republic of Korea
5 shared publications
Latest Publications
- Open Access
- 15 April 2026
- Published by Korean Cancer Association in Cancer Research and Treatment
- Vol. 58 (2) , 481-491
Secondary Cancer Risk in Breast Cancer with and without Radiotherapy: The Observational Health Data Sciences and Informatics (OHDSI) Cohort Study
PurposeRadiotherapy is used to reduce the risk of breast cancer recurrence after surgery, but it is a potential cause of secondary cancer. We validated the risk of secondary cancer in primary breast cancer who received radiotherapy compared with those who did not from a matched cohort using a large-scale observational study of the Observational Health Data Sciences and Informatics (OHDSI) data network.Materials and MethodsA retrospective comparative cohort study using propensity score-matched cohorts was performed using two Observational Medical Outcome Partnership common data model databases, from tertiary general hospitals in South Korea. Among female patients who underwent surgery after the diagnosis of breast cancer, the risk of secondary primary malignant occurrence after 1:1 matching was analyzed.ResultsAmong 27,078 patients with breast cancer, there was no significant difference in the risk of secondary cancer following radiotherapy in 4,426 patients after 1:1 propensity-score matching. Further, there were no significant differences in the sensitivity analysis according to age, latency period, and number of radiation treatments.ConclusionThere was no difference in the risk of secondary cancer in the patients diagnosed with breast cancer depending on whether or not radiotherapy was performed after surgery. In the future, it is necessary to analyze including data generated during cancer treatment.
- 15 April 2026
- Published by Korean Cancer Association in Cancer Research and Treatment
- Vol. 58 (2) , 481-491
Open AccessSecondary Cancer Risk in Breast Cancer with and without Radiotherapy: The Observational Health Data Sciences and Informatics (OHDSI) Cohort Study
PurposeRadiotherapy is used to reduce the risk of breast cancer recurrence after surgery, but it is a potential cause of secondary cancer. We validated the risk of secondary cancer in primary breast cancer who received radiotherapy compared with those who did not from a matched cohort using a large-scale observational study of the Observational Health Data Sciences and Informatics (OHDSI) data network.Materials and MethodsA retrospective comparative cohort study using propensity score-matched cohorts was performed using two Observational Medical Outcome Partnership common data model databases, from tertiary general hospitals in South Korea. Among female patients who underwent surgery after the diagnosis of breast cancer, the risk of secondary primary malignant occurrence after 1:1 matching was analyzed.ResultsAmong 27,078 patients with breast cancer, there was no significant difference in the risk of secondary cancer following radiotherapy in 4,426 patients after 1:1 propensity-score matching. Further, there were no significant differences in the sensitivity analysis according to age, latency period, and number of radiation treatments.ConclusionThere was no difference in the risk of secondary cancer in the patients diagnosed with breast cancer depending on whether or not radiotherapy was performed after surgery. In the future, it is necessary to analyze including data generated during cancer treatment.
- Open Access
- 15 May 2025
- Published by IOS Press
- Vol. 327, 611-612
Conversion of Nursing Statements into the OMOP Common Data Model
The aim of this study is to convert nursing statements into the OMOP CDM for use in observational studies. We mapped nursing statements to SNOMED CT concepts and converted them into the OMOP CDM format through an ETL process. As a result, approximately 200 million nursing statements were loaded into the CDM’s OBSERVATION, PROCEDURE_OCCURRENCE, and MEASUREMENT tables.
- 15 May 2025
- Published by IOS Press
- Vol. 327, 611-612
Open AccessConversion of Nursing Statements into the OMOP Common Data Model
The aim of this study is to convert nursing statements into the OMOP CDM for use in observational studies. We mapped nursing statements to SNOMED CT concepts and converted them into the OMOP CDM format through an ETL process. As a result, approximately 200 million nursing statements were loaded into the CDM’s OBSERVATION, PROCEDURE_OCCURRENCE, and MEASUREMENT tables.
- Open Access
- 9 April 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e66366
Developing a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study
Background: Considering that most patients with low or no significant risk factors can safely undergo noncardiac surgery without additional cardiac evaluation, and given the excessive evaluations often performed in patients undergoing intermediate or higher risk noncardiac surgeries, practical preoperative risk assessment tools are essential to reduce unnecessary delays for urgent outpatient services and manage medical costs more efficiently. Objective: This study aimed to use the Observational Medical Outcomes Partnership Common Data Model to develop a predictive model by applying machine learning algorithms that can effectively predict major adverse cardiac and cerebrovascular events (MACCE) in patients undergoing noncardiac surgery. Methods: This retrospective observational network study collected data by converting electronic health records into a standardized Observational Medical Outcomes Partnership Common Data Model format. The study was conducted in 2 tertiary hospitals. Data included demographic information, diagnoses, laboratory results, medications, surgical types, and clinical outcomes. A total of 46,225 patients were recruited from Seoul National University Bundang Hospital and 396,424 from Asan Medical Center. We selected patients aged 65 years and older undergoing noncardiac surgeries, excluding cardiac or emergency surgeries, and those with less than 30 days of observation. Using these observational health care data, we developed machine learning–based prediction models using the observational health data sciences and informatics open-source patient-level prediction package in R (version 4.1.0; R Foundation for Statistical Computing). A total of 5 machine learning algorithms, including random forest, were developed and validated internally and externally, with performance assessed through the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve, and calibration plots. Results: All machine learning prediction models surpassed the Revised Cardiac Risk Index in MACCE prediction performance (AUROC=0.704). Random forest showed the best results, achieving AUROC values of 0.897 (95% CI 0.883-0.911) internally and 0.817 (95% CI 0.815-0.819) externally, with an area under the precision-recall curve of 0.095. Among 46,225 patients of the Seoul National University Bundang Hospital, MACCE occurred in 4.9% (2256/46,225), including myocardial infarction (907/46,225, 2%) and stroke (799/46,225, 1.7%), while in-hospital mortality was 0.9% (419/46,225). For Asan Medical Center, 6.3% (24,861/396,424) of patients experienced MACCE, with 1.5% (6017/396,424) stroke and 3% (11,875/396,424) in-hospital mortality. Furthermore, the significance of predictors linked to previous diagnoses and laboratory measurements underscored their critical role in effectively predicting perioperative risk. Conclusions: Our prediction models outperformed the widely used Revised Cardiac Risk Index in predicting MACCE within 30 days after noncardiac surgery, demonstrating superior calibration and generalizability across institutions. Its use can optimize preoperative evaluations, minimize unnecessary testing, and streamline perioperative care, significantly improving patient outcomes and resource use. We anticipate that applying this model to actual electronic health records will benefit clinical practice.
- 9 April 2025
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 27, e66366
Open AccessDeveloping a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study
Background: Considering that most patients with low or no significant risk factors can safely undergo noncardiac surgery without additional cardiac evaluation, and given the excessive evaluations often performed in patients undergoing intermediate or higher risk noncardiac surgeries, practical preoperative risk assessment tools are essential to reduce unnecessary delays for urgent outpatient services and manage medical costs more efficiently. Objective: This study aimed to use the Observational Medical Outcomes Partnership Common Data Model to develop a predictive model by applying machine learning algorithms that can effectively predict major adverse cardiac and cerebrovascular events (MACCE) in patients undergoing noncardiac surgery. Methods: This retrospective observational network study collected data by converting electronic health records into a standardized Observational Medical Outcomes Partnership Common Data Model format. The study was conducted in 2 tertiary hospitals. Data included demographic information, diagnoses, laboratory results, medications, surgical types, and clinical outcomes. A total of 46,225 patients were recruited from Seoul National University Bundang Hospital and 396,424 from Asan Medical Center. We selected patients aged 65 years and older undergoing noncardiac surgeries, excluding cardiac or emergency surgeries, and those with less than 30 days of observation. Using these observational health care data, we developed machine learning–based prediction models using the observational health data sciences and informatics open-source patient-level prediction package in R (version 4.1.0; R Foundation for Statistical Computing). A total of 5 machine learning algorithms, including random forest, were developed and validated internally and externally, with performance assessed through the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve, and calibration plots. Results: All machine learning prediction models surpassed the Revised Cardiac Risk Index in MACCE prediction performance (AUROC=0.704). Random forest showed the best results, achieving AUROC values of 0.897 (95% CI 0.883-0.911) internally and 0.817 (95% CI 0.815-0.819) externally, with an area under the precision-recall curve of 0.095. Among 46,225 patients of the Seoul National University Bundang Hospital, MACCE occurred in 4.9% (2256/46,225), including myocardial infarction (907/46,225, 2%) and stroke (799/46,225, 1.7%), while in-hospital mortality was 0.9% (419/46,225). For Asan Medical Center, 6.3% (24,861/396,424) of patients experienced MACCE, with 1.5% (6017/396,424) stroke and 3% (11,875/396,424) in-hospital mortality. Furthermore, the significance of predictors linked to previous diagnoses and laboratory measurements underscored their critical role in effectively predicting perioperative risk. Conclusions: Our prediction models outperformed the widely used Revised Cardiac Risk Index in predicting MACCE within 30 days after noncardiac surgery, demonstrating superior calibration and generalizability across institutions. Its use can optimize preoperative evaluations, minimize unnecessary testing, and streamline perioperative care, significantly improving patient outcomes and resource use. We anticipate that applying this model to actual electronic health records will benefit clinical practice.
- Open Access
- 20 December 2024
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 12 (1) , e67056
Automated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study
Background: Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. Objective: This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. Methods: We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. Results: Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. Conclusions: The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types.
- 20 December 2024
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 12 (1) , e67056
Open AccessAutomated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study
Background: Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. Objective: This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. Methods: We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. Results: Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. Conclusions: The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types.
- Open Access
- 22 November 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e59260
Hospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study
Background: Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. Objective: In this study, we developed and validated a machine learning (ML)–based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). Methods: Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. Results: The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). Conclusions: The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings.
- 22 November 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e59260
Open AccessHospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study
Background: Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. Objective: In this study, we developed and validated a machine learning (ML)–based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). Methods: Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. Results: The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). Conclusions: The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings.
- Open Access
- 8 October 2024
- Published by JMIR Publications Inc. in JMIR Public Health and Surveillance
- Vol. 10 (1) , e56741
Patient-Centered Economic Burden of Diabetic Macular Edema: Retrospective Cohort Study
Background: Diabetic macular edema (DME), a leading cause of blindness, requires treatment with costly drugs, such as anti–vascular endothelial growth factor (VEGF) agents. The prolonged use of these effective but expensive drugs results in an incremental economic burden for patients with DME compared with those with diabetes mellitus (DM) without DME. However, there are no studies on the long-term patient-centered economic burden of DME after reimbursement for anti-VEGFs. Objective: This retrospective cohort study aims to estimate the 3-year patient-centered economic burden of DME compared with DM without DME, using the Common Data Model. Methods: We used medical data from 1,903,603 patients (2003-2020), transformed and validated using the Observational Medical Outcomes Partnership Common Data Model from Seoul National University Bundang Hospital. We defined the group with DME as patients aged >18 years with nonproliferative diabetic retinopathy and intravitreal anti-VEGF or steroid prescriptions. As control, we defined the group with DM without DME as patients aged >18 years with DM or diabetic retinopathy without intravitreal anti-VEGF or steroid prescriptions. Propensity score matching, performed using a regularized logistic regression with a Laplace prior, addressed selection bias. We estimated direct medical costs over 3 years categorized into total costs, reimbursement costs, nonreimbursement costs, out-of-pocket costs, and costs covered by insurance, as well as healthcare resource utilization. An exponential conditional model and a count model estimated unbiased incremental patient-centered economic burden using generalized linear models and a zero-inflation model. Results: In a cohort of 454 patients with DME matched with 1640 patients with DM, the economic burden of DME was significantly higher than that of DM, with total costs over 3 years being 2.09 (95% CI 1.78-2.47) times higher. Reimbursement costs were 1.89 (95% CI 1.57-2.28) times higher in the group with DME than with the group with DM, while nonreimbursement costs were 2.54 (95% CI 2.12-3.06) times higher. Out-of-pocket costs and costs covered by insurance were also higher by a factor of 2.11 (95% CI 1.58-2.59) and a factor of 2.01 (95% CI 1.85-2.42), respectively. Patients with DME had a significantly higher number of outpatient (1.87-fold) and inpatient (1.99-fold) visits compared with those with DM (P<.001 in all cases). Conclusions: Patients with DME experience a heightened economic burden compared with diabetic patients without DME. The substantial and enduring economic impact observed in real-world settings underscores the need to alleviate patients’ burden through preventive measures, effective management, appropriate reimbursement policies, and the development of innovative treatments. Strategies to mitigate the economic impact of DME should include proactive approaches such as expanding anti-VEGF reimbursement criteria, approving and reimbursing cost-effective drugs such as bevacizumab, advocating for proactive eye examinations, and embracing early diagnosis by ophthalmologists facilitated by cutting-edge methodologies such as artificial intelligence for patients with DM.
- 8 October 2024
- Published by JMIR Publications Inc. in JMIR Public Health and Surveillance
- Vol. 10 (1) , e56741
Open AccessPatient-Centered Economic Burden of Diabetic Macular Edema: Retrospective Cohort Study
Background: Diabetic macular edema (DME), a leading cause of blindness, requires treatment with costly drugs, such as anti–vascular endothelial growth factor (VEGF) agents. The prolonged use of these effective but expensive drugs results in an incremental economic burden for patients with DME compared with those with diabetes mellitus (DM) without DME. However, there are no studies on the long-term patient-centered economic burden of DME after reimbursement for anti-VEGFs. Objective: This retrospective cohort study aims to estimate the 3-year patient-centered economic burden of DME compared with DM without DME, using the Common Data Model. Methods: We used medical data from 1,903,603 patients (2003-2020), transformed and validated using the Observational Medical Outcomes Partnership Common Data Model from Seoul National University Bundang Hospital. We defined the group with DME as patients aged >18 years with nonproliferative diabetic retinopathy and intravitreal anti-VEGF or steroid prescriptions. As control, we defined the group with DM without DME as patients aged >18 years with DM or diabetic retinopathy without intravitreal anti-VEGF or steroid prescriptions. Propensity score matching, performed using a regularized logistic regression with a Laplace prior, addressed selection bias. We estimated direct medical costs over 3 years categorized into total costs, reimbursement costs, nonreimbursement costs, out-of-pocket costs, and costs covered by insurance, as well as healthcare resource utilization. An exponential conditional model and a count model estimated unbiased incremental patient-centered economic burden using generalized linear models and a zero-inflation model. Results: In a cohort of 454 patients with DME matched with 1640 patients with DM, the economic burden of DME was significantly higher than that of DM, with total costs over 3 years being 2.09 (95% CI 1.78-2.47) times higher. Reimbursement costs were 1.89 (95% CI 1.57-2.28) times higher in the group with DME than with the group with DM, while nonreimbursement costs were 2.54 (95% CI 2.12-3.06) times higher. Out-of-pocket costs and costs covered by insurance were also higher by a factor of 2.11 (95% CI 1.58-2.59) and a factor of 2.01 (95% CI 1.85-2.42), respectively. Patients with DME had a significantly higher number of outpatient (1.87-fold) and inpatient (1.99-fold) visits compared with those with DM (P<.001 in all cases). Conclusions: Patients with DME experience a heightened economic burden compared with diabetic patients without DME. The substantial and enduring economic impact observed in real-world settings underscores the need to alleviate patients’ burden through preventive measures, effective management, appropriate reimbursement policies, and the development of innovative treatments. Strategies to mitigate the economic impact of DME should include proactive approaches such as expanding anti-VEGF reimbursement criteria, approving and reimbursing cost-effective drugs such as bevacizumab, advocating for proactive eye examinations, and embracing early diagnosis by ophthalmologists facilitated by cutting-edge methodologies such as artificial intelligence for patients with DM.
- Open Access
- 30 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Automated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study (Preprint)
BACKGROUND Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. OBJECTIVE This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. METHODS We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual 8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. RESULTS Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. CONCLUSIONS The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types.
- 30 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAutomated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study (Preprint)
BACKGROUND Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. OBJECTIVE This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. METHODS We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual 8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. RESULTS Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. CONCLUSIONS The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types.
- Open Access
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Developing a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study (Preprint)
BACKGROUND Considering that most patients with low or no significant risk factors can safely undergo noncardiac surgery without additional cardiac evaluation, and given the excessive evaluations often performed in patients undergoing intermediate or higher risk noncardiac surgeries, practical preoperative risk assessment tools are essential to reduce unnecessary delays for urgent outpatient services and manage medical costs more efficiently. OBJECTIVE This study aimed to use the Observational Medical Outcomes Partnership Common Data Model to develop a predictive model by applying machine learning algorithms that can effectively predict major adverse cardiac and cerebrovascular events (MACCE) in patients undergoing noncardiac surgery. METHODS This retrospective observational network study collected data by converting electronic health records into a standardized Observational Medical Outcomes Partnership Common Data Model format. The study was conducted in 2 tertiary hospitals. Data included demographic information, diagnoses, laboratory results, medications, surgical types, and clinical outcomes. A total of 46,225 patients were recruited from Seoul National University Bundang Hospital and 396,424 from Asan Medical Center. We selected patients aged 65 years and older undergoing noncardiac surgeries, excluding cardiac or emergency surgeries, and those with less than 30 days of observation. Using these observational health care data, we developed machine learning–based prediction models using the observational health data sciences and informatics open-source patient-level prediction package in R (version 4.1.0; R Foundation for Statistical Computing). A total of 5 machine learning algorithms, including random forest, were developed and validated internally and externally, with performance assessed through the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve, and calibration plots. RESULTS All machine learning prediction models surpassed the Revised Cardiac Risk Index in MACCE prediction performance (AUROC=0.704). Random forest showed the best results, achieving AUROC values of 0.897 (95% CI 0.883-0.911) internally and 0.817 (95% CI 0.815-0.819) externally, with an area under the precision-recall curve of 0.095. Among 46,225 patients of the Seoul National University Bundang Hospital, MACCE occurred in 4.9% (2256/46,225), including myocardial infarction (907/46,225, 2%) and stroke (799/46,225, 1.7%), while in-hospital mortality was 0.9% (419/46,225). For Asan Medical Center, 6.3% (24,861/396,424) of patients experienced MACCE, with 1.5% (6017/396,424) stroke and 3% (11,875/396,424) in-hospital mortality. Furthermore, the significance of predictors linked to previous diagnoses and laboratory measurements underscored their critical role in effectively predicting perioperative risk. CONCLUSIONS Our prediction models outperformed the widely used Revised Cardiac Risk Index in predicting MACCE within 30 days after noncardiac surgery, demonstrating superior calibration and generalizability across institutions. Its use can optimize preoperative evaluations, minimize unnecessary testing, and streamline perioperative care, significantly improving patient outcomes and resource use. We anticipate that applying this model to actual electronic health records will benefit clinical practice.
- 12 September 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDeveloping a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study (Preprint)
BACKGROUND Considering that most patients with low or no significant risk factors can safely undergo noncardiac surgery without additional cardiac evaluation, and given the excessive evaluations often performed in patients undergoing intermediate or higher risk noncardiac surgeries, practical preoperative risk assessment tools are essential to reduce unnecessary delays for urgent outpatient services and manage medical costs more efficiently. OBJECTIVE This study aimed to use the Observational Medical Outcomes Partnership Common Data Model to develop a predictive model by applying machine learning algorithms that can effectively predict major adverse cardiac and cerebrovascular events (MACCE) in patients undergoing noncardiac surgery. METHODS This retrospective observational network study collected data by converting electronic health records into a standardized Observational Medical Outcomes Partnership Common Data Model format. The study was conducted in 2 tertiary hospitals. Data included demographic information, diagnoses, laboratory results, medications, surgical types, and clinical outcomes. A total of 46,225 patients were recruited from Seoul National University Bundang Hospital and 396,424 from Asan Medical Center. We selected patients aged 65 years and older undergoing noncardiac surgeries, excluding cardiac or emergency surgeries, and those with less than 30 days of observation. Using these observational health care data, we developed machine learning–based prediction models using the observational health data sciences and informatics open-source patient-level prediction package in R (version 4.1.0; R Foundation for Statistical Computing). A total of 5 machine learning algorithms, including random forest, were developed and validated internally and externally, with performance assessed through the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve, and calibration plots. RESULTS All machine learning prediction models surpassed the Revised Cardiac Risk Index in MACCE prediction performance (AUROC=0.704). Random forest showed the best results, achieving AUROC values of 0.897 (95% CI 0.883-0.911) internally and 0.817 (95% CI 0.815-0.819) externally, with an area under the precision-recall curve of 0.095. Among 46,225 patients of the Seoul National University Bundang Hospital, MACCE occurred in 4.9% (2256/46,225), including myocardial infarction (907/46,225, 2%) and stroke (799/46,225, 1.7%), while in-hospital mortality was 0.9% (419/46,225). For Asan Medical Center, 6.3% (24,861/396,424) of patients experienced MACCE, with 1.5% (6017/396,424) stroke and 3% (11,875/396,424) in-hospital mortality. Furthermore, the significance of predictors linked to previous diagnoses and laboratory measurements underscored their critical role in effectively predicting perioperative risk. CONCLUSIONS Our prediction models outperformed the widely used Revised Cardiac Risk Index in predicting MACCE within 30 days after noncardiac surgery, demonstrating superior calibration and generalizability across institutions. Its use can optimize preoperative evaluations, minimize unnecessary testing, and streamline perioperative care, significantly improving patient outcomes and resource use. We anticipate that applying this model to actual electronic health records will benefit clinical practice.
- Open Access
- 12 July 2024
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 12, e59187
Integrating Clinical Data and Medical Imaging in Lung Cancer: Feasibility Study Using the Observational Medical Outcomes Partnership Common Data Model Extension
Background: Digital transformation, particularly the integration of medical imaging with clinical data, is vital in personalized medicine. The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) standardizes health data. However, integrating medical imaging remains a challenge. Objective: This study proposes a method for combining medical imaging data with the OMOP CDM to improve multimodal research. Methods: Our approach included the analysis and selection of digital imaging and communications in medicine header tags, validation of data formats, and alignment according to the OMOP CDM framework. The Fast Healthcare Interoperability Resources ImagingStudy profile guided our consistency in column naming and definitions. Imaging Common Data Model (I-CDM), constructed using the entity-attribute-value model, facilitates scalable and efficient medical imaging data management. For patients with lung cancer diagnosed between 2010 and 2017, we introduced 4 new tables—IMAGING_STUDY, IMAGING_SERIES, IMAGING_ANNOTATION, and FILEPATH—to standardize various imaging-related data and link to clinical data. Results: This framework underscores the effectiveness of I-CDM in enhancing our understanding of lung cancer diagnostics and treatment strategies. The implementation of the I-CDM tables enabled the structured organization of a comprehensive data set, including 282,098 IMAGING_STUDY, 5,674,425 IMAGING_SERIES, and 48,536 IMAGING_ANNOTATION records, illustrating the extensive scope and depth of the approach. A scenario-based analysis using actual data from patients with lung cancer underscored the feasibility of our approach. A data quality check applying 44 specific rules confirmed the high integrity of the constructed data set, with all checks successfully passed, underscoring the reliability of our findings. Conclusions: These findings indicate that I-CDM can improve the integration and analysis of medical imaging and clinical data. By addressing the challenges in data standardization and management, our approach contributes toward enhancing diagnostics and treatment strategies. Future research should expand the application of I-CDM to diverse disease populations and explore its wide-ranging utility for medical conditions.
- 12 July 2024
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 12, e59187
Open AccessIntegrating Clinical Data and Medical Imaging in Lung Cancer: Feasibility Study Using the Observational Medical Outcomes Partnership Common Data Model Extension
Background: Digital transformation, particularly the integration of medical imaging with clinical data, is vital in personalized medicine. The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) standardizes health data. However, integrating medical imaging remains a challenge. Objective: This study proposes a method for combining medical imaging data with the OMOP CDM to improve multimodal research. Methods: Our approach included the analysis and selection of digital imaging and communications in medicine header tags, validation of data formats, and alignment according to the OMOP CDM framework. The Fast Healthcare Interoperability Resources ImagingStudy profile guided our consistency in column naming and definitions. Imaging Common Data Model (I-CDM), constructed using the entity-attribute-value model, facilitates scalable and efficient medical imaging data management. For patients with lung cancer diagnosed between 2010 and 2017, we introduced 4 new tables—IMAGING_STUDY, IMAGING_SERIES, IMAGING_ANNOTATION, and FILEPATH—to standardize various imaging-related data and link to clinical data. Results: This framework underscores the effectiveness of I-CDM in enhancing our understanding of lung cancer diagnostics and treatment strategies. The implementation of the I-CDM tables enabled the structured organization of a comprehensive data set, including 282,098 IMAGING_STUDY, 5,674,425 IMAGING_SERIES, and 48,536 IMAGING_ANNOTATION records, illustrating the extensive scope and depth of the approach. A scenario-based analysis using actual data from patients with lung cancer underscored the feasibility of our approach. A data quality check applying 44 specific rules confirmed the high integrity of the constructed data set, with all checks successfully passed, underscoring the reliability of our findings. Conclusions: These findings indicate that I-CDM can improve the integration and analysis of medical imaging and clinical data. By addressing the challenges in data standardization and management, our approach contributes toward enhancing diagnostics and treatment strategies. Future research should expand the application of I-CDM to diverse disease populations and explore its wide-ranging utility for medical conditions.
- Open Access
- 7 April 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Hospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study (Preprint)
BACKGROUND Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. OBJECTIVE In this study, we developed and validated a machine learning (ML)–based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). METHODS Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. RESULTS The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). CONCLUSIONS The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings.
- 7 April 2024
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessHospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study (Preprint)
BACKGROUND Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. OBJECTIVE In this study, we developed and validated a machine learning (ML)–based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). METHODS Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. RESULTS The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). CONCLUSIONS The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings.