Jia Li
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Jia Li has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
Cao Feng
Xuanwu Hospital, Capital Medical University, Beijing, China
1 shared publication- Chengbin Hou
School of Computing and Artificial Intelligence, Fuyao University of Science and Technology, Fuzhou, China
1 shared publication Ding Yixuan
Xuanwu Hospital, Capital Medical University, Beijing, China
1 shared publicationFangyan Liu
Department of Emergency, National Clinical Research Center for Geriatric Diseases, Xuanwu Hospital of Capital Medical University, Beijing, China
1 shared publication- Fei Xiaolu
Xuanwu Hospital, Capital Medical University, Beijing, China
1 shared publication - Hairong Lv
Tsinghua University
1 shared publication
Latest Publications
- Open Access
- 23 September 2026
- Published by Springer Nature in Journal of Medical Systems
- Vol. 50 (1) , 136
Development of a Framework for Evaluating Large Language Model Safety and Reliability: a Proof-of-Concept Evaluation
Large language models (LLMs) are entering clinical decision support faster than methodology can characterise their safety. Aggregate accuracy treats all errors as interchangeable and cannot support safe deployment under Software as a Medical Device (SaMD) and EU AI Act frameworks. To develop and demonstrate a framework for evaluating large language model safety and reliability using catastrophic-failure frequency and response reproducibility, incorporating a pre-specified error taxonomy, difficulty-stratified analysis, and two-layer response consistency. The framework was applied to 54 diagnostically challenging emergency cases from non-public institutional records. Six LLMs were accessed via application programming interfaces (APIs) (text-only, zero-shot, defaults; August–December 2025) and queried three times each, producing 972 physician-scored responses. Aggregate scores concealed heterogeneity: anchoring-bias failure (score ≤ 3) varied nearly four-fold (6.9%–26.4%) and rare-disease recognition six-fold (6.1%–37.9%); catastrophic (dangerous-recommendation, score ≤ 2) rates were two-tiered (1.5% for the safest two vs. 6.0% for the rest; p < 0.001), though within-tier ranks were inseparable (2–12 per model). Despite a middle-tier mean, GPT-5.1 was least reproducible (within-case SD 2.69) and fell within the higher catastrophic-failure tier; across models, 31 of 34 dangerous combinations were stochastic, not systematic—a failure mode hidden by aggregate metrics. Open-source DeepSeek R1 was statistically equivalent to Gemini 3 Pro within a 1.5-point margin by two one-sided tests (TOST: Δ − 0.09; 90% CI − 0.47 to 0.30; p < 0.001). In this single-centre proof-of-concept, aggregate accuracy was insufficient for safety characterisation: models with indistinguishable mean accuracy carried different catastrophic-failure tiers and reproducibility profiles. Error-type and response-consistency profiling may inform model selection, ensemble design, and conformity assessment.
- 23 September 2026
- Published by Springer Nature in Journal of Medical Systems
- Vol. 50 (1) , 136
Open AccessDevelopment of a Framework for Evaluating Large Language Model Safety and Reliability: a Proof-of-Concept Evaluation
Large language models (LLMs) are entering clinical decision support faster than methodology can characterise their safety. Aggregate accuracy treats all errors as interchangeable and cannot support safe deployment under Software as a Medical Device (SaMD) and EU AI Act frameworks. To develop and demonstrate a framework for evaluating large language model safety and reliability using catastrophic-failure frequency and response reproducibility, incorporating a pre-specified error taxonomy, difficulty-stratified analysis, and two-layer response consistency. The framework was applied to 54 diagnostically challenging emergency cases from non-public institutional records. Six LLMs were accessed via application programming interfaces (APIs) (text-only, zero-shot, defaults; August–December 2025) and queried three times each, producing 972 physician-scored responses. Aggregate scores concealed heterogeneity: anchoring-bias failure (score ≤ 3) varied nearly four-fold (6.9%–26.4%) and rare-disease recognition six-fold (6.1%–37.9%); catastrophic (dangerous-recommendation, score ≤ 2) rates were two-tiered (1.5% for the safest two vs. 6.0% for the rest; p < 0.001), though within-tier ranks were inseparable (2–12 per model). Despite a middle-tier mean, GPT-5.1 was least reproducible (within-case SD 2.69) and fell within the higher catastrophic-failure tier; across models, 31 of 34 dangerous combinations were stochastic, not systematic—a failure mode hidden by aggregate metrics. Open-source DeepSeek R1 was statistically equivalent to Gemini 3 Pro within a 1.5-point margin by two one-sided tests (TOST: Δ − 0.09; 90% CI − 0.47 to 0.30; p < 0.001). In this single-centre proof-of-concept, aggregate accuracy was insufficient for safety characterisation: models with indistinguishable mean accuracy carried different catastrophic-failure tiers and reproducibility profiles. Error-type and response-consistency profiling may inform model selection, ensemble design, and conformity assessment.
- Open Access
- 28 February 2026
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Predicting infected pancreatic necrosis in acute pancreatitis using machine learning models and feature selection
Infected pancreatic necrosis (IPN) is a life-threatening complication of acute pancreatitis (AP), and its early prediction remains challenging. This study aimed to develop and externally validate interpretable machine learning models for individualized IPN risk prediction. A total of 728 patients with AP admitted to Xuanwu Hospital, Capital Medical University, between 2017 and 2023 were retrospectively analyzed. Embedded feature selection was incorporated within model training using regularized linear and tree-based algorithms to enhance interpretability and prevent overfitting. Five machine learning algorithms and one neural network model were evaluated through nested cross-validation and an independent temporal external cohort consisting of 166 AP patients admitted to Xuanwu Hospital, Capital Medical University, between 2022 and 2023. Model discrimination, precision–recall, and probability calibration were assessed, and model explainability was analyzed using Shapley Additive Explanations (SHAP). The Random Forest model achieved the best overall performance, achieving an external AUC of 0.764 (95% CI 0.696–0.830, \(P < 0.001\)), precision of 0.893, recall of 0.604, and the lowest Brier score, indicating reliable probability calibration. SHAP analysis identified Fibrinogen, APACHE II score, D-dimer, IL-6, and C-reactive protein as key predictors associated with increased IPN risk, while higher Lymphocyte count, and Hematocrit were protective. These findings are consistent clinical pathophysiology. The interpretable Random Forest model demonstrated robust discrimination and calibration for IPN prediction, providing a transparent and data-driven framework for early risk stratification in acute pancreatitis. Prospective multicenter validation is warranted before clinical implementation.- 28 February 2026
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Open AccessPredicting infected pancreatic necrosis in acute pancreatitis using machine learning models and feature selection
Infected pancreatic necrosis (IPN) is a life-threatening complication of acute pancreatitis (AP), and its early prediction remains challenging. This study aimed to develop and externally validate interpretable machine learning models for individualized IPN risk prediction. A total of 728 patients with AP admitted to Xuanwu Hospital, Capital Medical University, between 2017 and 2023 were retrospectively analyzed. Embedded feature selection was incorporated within model training using regularized linear and tree-based algorithms to enhance interpretability and prevent overfitting. Five machine learning algorithms and one neural network model were evaluated through nested cross-validation and an independent temporal external cohort consisting of 166 AP patients admitted to Xuanwu Hospital, Capital Medical University, between 2022 and 2023. Model discrimination, precision–recall, and probability calibration were assessed, and model explainability was analyzed using Shapley Additive Explanations (SHAP). The Random Forest model achieved the best overall performance, achieving an external AUC of 0.764 (95% CI 0.696–0.830, \(P < 0.001\)), precision of 0.893, recall of 0.604, and the lowest Brier score, indicating reliable probability calibration. SHAP analysis identified Fibrinogen, APACHE II score, D-dimer, IL-6, and C-reactive protein as key predictors associated with increased IPN risk, while higher Lymphocyte count, and Hematocrit were protective. These findings are consistent clinical pathophysiology. The interpretable Random Forest model demonstrated robust discrimination and calibration for IPN prediction, providing a transparent and data-driven framework for early risk stratification in acute pancreatitis. Prospective multicenter validation is warranted before clinical implementation. - 12 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 75 (00189529) , 709-722
KGC-Explainer: Toward Explainable Knowledge Graph Completion
Knowledge Graph Completion (KGC) aims to infer missing triples for a given knowledge graph, which can be adopted to various fields ranging from scientific research to real-world applications. Despite the success of numerous KGC methods, the lack of explainability remains a common drawback. However, the explanation of KGC results is crucial in many cases such as providing medical diagnosis and recommending candidates for costly experiments, which could increase the reliability of these techniques to humans. Although a few explainable KGC methods have been proposed, the explainability module is designed for the specific KGC model and cannot be utilized for state-of-the-art KGC models. In this work, we present a post-hoc generic method, namely KGC-Explainer, that can be applied to any KGC model providing triplet scores. KGC-Explainer not only incorporates the input KGC model itself, but also leverages the structural and textual information in knowledge graphs. To demonstrate KGC-Explainer achieving design goals and the superiority of it over other methods, we conduct extensive experiments on two real-world knowledge graphs with different domains and languages, one coming from medical domain in Chinese and another coming from general domain in English. Additionally, we compare the KGC explanation with human explanation, showcasing the practical significance of KGC-Explainer.
- 12 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Reliability
- Vol. 75 (00189529) , 709-722
KGC-Explainer: Toward Explainable Knowledge Graph Completion
Knowledge Graph Completion (KGC) aims to infer missing triples for a given knowledge graph, which can be adopted to various fields ranging from scientific research to real-world applications. Despite the success of numerous KGC methods, the lack of explainability remains a common drawback. However, the explanation of KGC results is crucial in many cases such as providing medical diagnosis and recommending candidates for costly experiments, which could increase the reliability of these techniques to humans. Although a few explainable KGC methods have been proposed, the explainability module is designed for the specific KGC model and cannot be utilized for state-of-the-art KGC models. In this work, we present a post-hoc generic method, namely KGC-Explainer, that can be applied to any KGC model providing triplet scores. KGC-Explainer not only incorporates the input KGC model itself, but also leverages the structural and textual information in knowledge graphs. To demonstrate KGC-Explainer achieving design goals and the superiority of it over other methods, we conduct extensive experiments on two real-world knowledge graphs with different domains and languages, one coming from medical domain in Chinese and another coming from general domain in English. Additionally, we compare the KGC explanation with human explanation, showcasing the practical significance of KGC-Explainer.