Cheng Bi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Cheng Bi has a total of 61 co-authors. Here are the top co-authors by number of shared publications:
- Bairong Shen
Department of Respiratory and Critical Care Medicine and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, China
13 shared publications - Xin Zheng
Department of Ophthalmology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital of Sichuan University, No.2222, Xinchuan Road, Gaoxin District, Chengdu, Sichuan, 610041, China, 86 15995854635
9 shared publications - Jie Song
Department of Ophthalmology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital of Sichuan University, No.2222, Xinchuan Road, Gaoxin District, Chengdu, Sichuan, 610041, China, 86 15995854635
8 shared publications - Yuxin Zhang
Department of Ophthalmology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular Network, West China Hospital of Sichuan University, No.2222, Xinchuan Road, Gaoxin District, Chengdu, Sichuan, 610041, China, 86 15995854635
8 shared publications - Zhichuan Xu
Department of Medical Oncology, Institutes for Systems Genetics, Frontiers Science Center for Disease-Related Molecular Network, West China Hospital, Sichuan University, No 2222 Xinchuan Road, Gaoxin District, Chengdu, Sichuan, 610000, China, 86 15995854635, 86 28 61528682
7 shared publications - Dan Cao
West China Hospital 1 Division of Abdominal Tumor, Department of Medical Oncology, Cancer Center and State Key Laboratory of Biological Therapy , Chengdu, China
3 shared publications
Latest Publications
- Open Access
- 2 November 2026
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 14, e105860
Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study
Background: Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear. Objective: This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performance. Methods: This diagnostic accuracy study used 4746 facial images from the GestaltMatcher Database (GMDB), including 882 images in GMDB-Frequent from diseases with at least 6 images and 3864 images in GMDB-Rare from diseases with fewer than 6 images. Three locally deployed open-weight MLLMs—Gemma-4-31B, Qwen3.6-35B-A3B, and Qwen3.8-27B—generated 50 ranked candidate diagnoses under raw image, CodeFormer-enhanced, Qwen-Image-Edit-2511–generated red-circle–prompted, mismatched red-circle, and 2-stage image-to-phenotype conditions. Candidate diagnoses were mapped to Mondo Disease Ontology (MONDO) identifiers. In addition, we evaluated a random baseline, visual-prompt quality, expert-annotated regions, repeated-inference stability, model-reported confidence, diagnostic-list perplexity, prediction overlap, and disease composition. Results: Top-50 accuracy ranged from 13.95% to 18.25% in GMDB-Frequent and from 0.57% to 1.35% in GMDB-Rare. Red-circle prompting produced numerical accuracy increases in 5 of 6 model-subset comparisons. Qwen3.8-27B showed a significant improvement in GMDB-Rare, with accuracy increasing from 0.57% to 1.27% (adjusted P=.009). In the good visual-prompt subgroup of GMDB-Frequent, Gemma-4-31B showed significant improvements over the super-resolution condition in both physician-rated analyses, with accuracy increases of 5.76 percentage points (Holm-adjusted P=.005) and 4.46 percentage points (Holm-adjusted P=.048), respectively. Matched red-circle inputs yielded higher accuracy than mismatched inputs across all model-subset comparisons, with adjusted P>.05. The image-to-phenotype workflow improved accuracy for Gemma-4-31B in GMDB-Frequent and Qwen3.8-27B in GMDB-Rare. In GMDB-Rare, the random baseline achieved 8.15% accuracy and exceeded all 3 models under the raw image and red-circle conditions. Prediction sets remained substantially overlapping after patient-image replacement, and top-1 predictions concentrated on a small number of diseases. Conclusions: Under the evaluated experimental settings, the 3 MLLMs showed limited ability to diagnose rare genetic diseases from facial images, particularly diseases in GMDB-Rare. Visual prompting and image-to-phenotype diagnosis produced benefits in specific models and subgroups. The prediction-overlap, mismatched-image, and disease-composition analyses indicate that model outputs were only weakly constrained by patient-specific facial information. Further studies should evaluate physician comparisons, prospective multicenter data, and phenotype-grounded multimodal diagnostic workflows.
- 2 November 2026
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 14, e105860
Open AccessDiagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study
Background: Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear. Objective: This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performance. Methods: This diagnostic accuracy study used 4746 facial images from the GestaltMatcher Database (GMDB), including 882 images in GMDB-Frequent from diseases with at least 6 images and 3864 images in GMDB-Rare from diseases with fewer than 6 images. Three locally deployed open-weight MLLMs—Gemma-4-31B, Qwen3.6-35B-A3B, and Qwen3.8-27B—generated 50 ranked candidate diagnoses under raw image, CodeFormer-enhanced, Qwen-Image-Edit-2511–generated red-circle–prompted, mismatched red-circle, and 2-stage image-to-phenotype conditions. Candidate diagnoses were mapped to Mondo Disease Ontology (MONDO) identifiers. In addition, we evaluated a random baseline, visual-prompt quality, expert-annotated regions, repeated-inference stability, model-reported confidence, diagnostic-list perplexity, prediction overlap, and disease composition. Results: Top-50 accuracy ranged from 13.95% to 18.25% in GMDB-Frequent and from 0.57% to 1.35% in GMDB-Rare. Red-circle prompting produced numerical accuracy increases in 5 of 6 model-subset comparisons. Qwen3.8-27B showed a significant improvement in GMDB-Rare, with accuracy increasing from 0.57% to 1.27% (adjusted P=.009). In the good visual-prompt subgroup of GMDB-Frequent, Gemma-4-31B showed significant improvements over the super-resolution condition in both physician-rated analyses, with accuracy increases of 5.76 percentage points (Holm-adjusted P=.005) and 4.46 percentage points (Holm-adjusted P=.048), respectively. Matched red-circle inputs yielded higher accuracy than mismatched inputs across all model-subset comparisons, with adjusted P>.05. The image-to-phenotype workflow improved accuracy for Gemma-4-31B in GMDB-Frequent and Qwen3.8-27B in GMDB-Rare. In GMDB-Rare, the random baseline achieved 8.15% accuracy and exceeded all 3 models under the raw image and red-circle conditions. Prediction sets remained substantially overlapping after patient-image replacement, and top-1 predictions concentrated on a small number of diseases. Conclusions: Under the evaluated experimental settings, the 3 MLLMs showed limited ability to diagnose rare genetic diseases from facial images, particularly diseases in GMDB-Rare. Visual prompting and image-to-phenotype diagnosis produced benefits in specific models and subgroups. The prediction-overlap, mismatched-image, and disease-composition analyses indicate that model outputs were only weakly constrained by patient-specific facial information. Further studies should evaluate physician comparisons, prospective multicenter data, and phenotype-grounded multimodal diagnostic workflows.
- Open Access
- 23 July 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e101711
Initial-Visit Specialty Triage in Rare Diseases Using Large Language Models: Retrospective Benchmarking Study
Background: Specialty triage at first contact is an overlooked step in early diagnostic pathways for rare diseases. Patients often present with overlapping, multisystem, and atypical manifestations, making first-visit specialty selection challenging and potentially prolonging diagnostic pathways. Objective: The aim of this study is to evaluate the accuracy, response time, and consistency of large language models (LLMs) for initial-visit specialty triage in rare diseases across multiple datasets, and to compare their performance with registered nurses and nonmedical participants. Methods: In this retrospective benchmarking study, we used 5 rare disease datasets: a publication-derived case set, 3 RareBench-derived datasets, and a Facial phenotype-Gene-Disease Dataset-derived set. Fourteen LLMs were evaluated over 5 independent runs per case. Performance was assessed using accuracy, response time, and consistency, with subgroup analyses by model accessibility, reasoning mode, parameter scale, and phenotype count. Human comparison was conducted on the publication-derived case set using registered nurses and nonmedical participants. Results: Across datasets, model accuracy ranged from 0.4378 to 0.7141. Claude-opus-4‐5 achieved the highest accuracy (0.7141) and consistency (0.9653), averaging 10.79 seconds per case. GPT-5.1 had the shortest response time (3.39 s/case) and high accuracy (0.6948). Proprietary models had numerically higher average accuracy than open-weight models (0.6973 vs 0.6365). Nonthinking models achieved higher average accuracy than thinking models (0.6789 vs 0.5826) and had shorter response times, although this exploratory comparison was based on a small number of thinking models. Accuracy varied by phenotype count, with higher performance in cases with 1 to 2 or more than 14 phenotypes. On the publication-derived case set, LLMs achieved higher average accuracy than registered nurses and nonmedical participants (0.5978 vs 0.4914 and 0.4573). Conclusions: LLMs showed potential as assistive tools for initial-visit specialty triage in rare diseases. Model choice, reasoning mode, and phenotype information density influenced performance, but subgroup findings should be interpreted cautiously. Future work should evaluate LLM-based specialty triage in prospective clinical settings and develop clinician-supervised workflows with traceable evidence support.
- 23 July 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e101711
Open AccessInitial-Visit Specialty Triage in Rare Diseases Using Large Language Models: Retrospective Benchmarking Study
Background: Specialty triage at first contact is an overlooked step in early diagnostic pathways for rare diseases. Patients often present with overlapping, multisystem, and atypical manifestations, making first-visit specialty selection challenging and potentially prolonging diagnostic pathways. Objective: The aim of this study is to evaluate the accuracy, response time, and consistency of large language models (LLMs) for initial-visit specialty triage in rare diseases across multiple datasets, and to compare their performance with registered nurses and nonmedical participants. Methods: In this retrospective benchmarking study, we used 5 rare disease datasets: a publication-derived case set, 3 RareBench-derived datasets, and a Facial phenotype-Gene-Disease Dataset-derived set. Fourteen LLMs were evaluated over 5 independent runs per case. Performance was assessed using accuracy, response time, and consistency, with subgroup analyses by model accessibility, reasoning mode, parameter scale, and phenotype count. Human comparison was conducted on the publication-derived case set using registered nurses and nonmedical participants. Results: Across datasets, model accuracy ranged from 0.4378 to 0.7141. Claude-opus-4‐5 achieved the highest accuracy (0.7141) and consistency (0.9653), averaging 10.79 seconds per case. GPT-5.1 had the shortest response time (3.39 s/case) and high accuracy (0.6948). Proprietary models had numerically higher average accuracy than open-weight models (0.6973 vs 0.6365). Nonthinking models achieved higher average accuracy than thinking models (0.6789 vs 0.5826) and had shorter response times, although this exploratory comparison was based on a small number of thinking models. Accuracy varied by phenotype count, with higher performance in cases with 1 to 2 or more than 14 phenotypes. On the publication-derived case set, LLMs achieved higher average accuracy than registered nurses and nonmedical participants (0.5978 vs 0.4914 and 0.4573). Conclusions: LLMs showed potential as assistive tools for initial-visit specialty triage in rare diseases. Model choice, reasoning mode, and phenotype information density influenced performance, but subgroup findings should be interpreted cautiously. Future work should evaluate LLM-based specialty triage in prospective clinical settings and develop clinician-supervised workflows with traceable evidence support.
- Open Access
- 30 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study (Preprint)
BACKGROUND Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear. OBJECTIVE This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performance. METHODS This diagnostic accuracy study used 4746 facial images from the GestaltMatcher Database (GMDB), including 882 images in GMDB-Frequent from diseases with at least 6 images and 3864 images in GMDB-Rare from diseases with fewer than 6 images. Three locally deployed open-weight MLLMs—Gemma-4-31B, Qwen3.6-35B-A3B, and Qwen3.8-27B—generated 50 ranked candidate diagnoses under raw image, CodeFormer-enhanced, Qwen-Image-Edit-2511–generated red-circle–prompted, mismatched red-circle, and 2-stage image-to-phenotype conditions. Candidate diagnoses were mapped to Mondo Disease Ontology (MONDO) identifiers. In addition, we evaluated a random baseline, visual-prompt quality, expert-annotated regions, repeated-inference stability, model-reported confidence, diagnostic-list perplexity, prediction overlap, and disease composition. RESULTS Top-50 accuracy ranged from 13.95% to 18.25% in GMDB-Frequent and from 0.57% to 1.35% in GMDB-Rare. Red-circle prompting produced numerical accuracy increases in 5 of 6 model-subset comparisons. Qwen3.8-27B showed a significant improvement in GMDB-Rare, with accuracy increasing from 0.57% to 1.27% (adjusted P=.009). In the good visual-prompt subgroup of GMDB-Frequent, Gemma-4-31B showed significant improvements over the super-resolution condition in both physician-rated analyses, with accuracy increases of 5.76 percentage points (Holm-adjusted P=.005) and 4.46 percentage points (Holm-adjusted P=.048), respectively. Matched red-circle inputs yielded higher accuracy than mismatched inputs across all model-subset comparisons, with adjusted P>.05. The image-to-phenotype workflow improved accuracy for Gemma-4-31B in GMDB-Frequent and Qwen3.8-27B in GMDB-Rare. In GMDB-Rare, the random baseline achieved 8.15% accuracy and exceeded all 3 models under the raw image and red-circle conditions. Prediction sets remained substantially overlapping after patient-image replacement, and top-1 predictions concentrated on a small number of diseases. CONCLUSIONS Under the evaluated experimental settings, the 3 MLLMs showed limited ability to diagnose rare genetic diseases from facial images, particularly diseases in GMDB-Rare. Visual prompting and image-to-phenotype diagnosis produced benefits in specific models and subgroups. The prediction-overlap, mismatched-image, and disease-composition analyses indicate that model outputs were only weakly constrained by patient-specific facial information. Further studies should evaluate physician comparisons, prospective multicenter data, and phenotype-grounded multimodal diagnostic workflows.
- 30 June 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessDiagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study (Preprint)
BACKGROUND Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear. OBJECTIVE This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performance. METHODS This diagnostic accuracy study used 4746 facial images from the GestaltMatcher Database (GMDB), including 882 images in GMDB-Frequent from diseases with at least 6 images and 3864 images in GMDB-Rare from diseases with fewer than 6 images. Three locally deployed open-weight MLLMs—Gemma-4-31B, Qwen3.6-35B-A3B, and Qwen3.8-27B—generated 50 ranked candidate diagnoses under raw image, CodeFormer-enhanced, Qwen-Image-Edit-2511–generated red-circle–prompted, mismatched red-circle, and 2-stage image-to-phenotype conditions. Candidate diagnoses were mapped to Mondo Disease Ontology (MONDO) identifiers. In addition, we evaluated a random baseline, visual-prompt quality, expert-annotated regions, repeated-inference stability, model-reported confidence, diagnostic-list perplexity, prediction overlap, and disease composition. RESULTS Top-50 accuracy ranged from 13.95% to 18.25% in GMDB-Frequent and from 0.57% to 1.35% in GMDB-Rare. Red-circle prompting produced numerical accuracy increases in 5 of 6 model-subset comparisons. Qwen3.8-27B showed a significant improvement in GMDB-Rare, with accuracy increasing from 0.57% to 1.27% (adjusted P=.009). In the good visual-prompt subgroup of GMDB-Frequent, Gemma-4-31B showed significant improvements over the super-resolution condition in both physician-rated analyses, with accuracy increases of 5.76 percentage points (Holm-adjusted P=.005) and 4.46 percentage points (Holm-adjusted P=.048), respectively. Matched red-circle inputs yielded higher accuracy than mismatched inputs across all model-subset comparisons, with adjusted P>.05. The image-to-phenotype workflow improved accuracy for Gemma-4-31B in GMDB-Frequent and Qwen3.8-27B in GMDB-Rare. In GMDB-Rare, the random baseline achieved 8.15% accuracy and exceeded all 3 models under the raw image and red-circle conditions. Prediction sets remained substantially overlapping after patient-image replacement, and top-1 predictions concentrated on a small number of diseases. CONCLUSIONS Under the evaluated experimental settings, the 3 MLLMs showed limited ability to diagnose rare genetic diseases from facial images, particularly diseases in GMDB-Rare. Visual prompting and image-to-phenotype diagnosis produced benefits in specific models and subgroups. The prediction-overlap, mismatched-image, and disease-composition analyses indicate that model outputs were only weakly constrained by patient-specific facial information. Further studies should evaluate physician comparisons, prospective multicenter data, and phenotype-grounded multimodal diagnostic workflows.
- Open Access
- 21 May 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e88614
Benchmarking Large Language Models and Prompt Engineering Strategies in Microsatellite Instability Cancers: Evaluation Study
Background: The reliability of general-purpose large language models (LLMs) for complex clinical tasks in specialized domains such as microsatellite instability (MSI) cancers remains critically uncharacterized. The absence of a domain-specific benchmark to evaluate and guide the optimization of their capabilities across diverse clinical tasks poses unevaluated risks to patient safety. Objective: This study aimed to develop and validate Microsatellite Instability Cancer Benchmark (MSIC-Bench), a novel, two-tiered benchmark for MSI cancer, covering both consensus and frontier knowledge. Using this framework, we aimed to systematically assess LLM performance across various prompting strategies, identify task-specific weaknesses, and reveal effective pathways for performance improvement. Methods: We developed MSIC-Bench, a 511-question benchmark derived from clinical guidelines and a curated knowledge base. Three state-of-the-art LLMs (GPT-4o, [OpenAI], Gemini 2.5 Pro [Google], and Claude Opus 4 [Anthropic]) were evaluated using 4 prompting strategies, including vanilla, chain-of-thought, reflection of thoughts, and retrieval-augmented generation (RAG), under both multiple-choice and open-ended modalities. Performance was assessed on accuracy, safety (honesty), error composition, and token usage. Results: LLMs demonstrated a significant “scaffolding effect,” with accuracy dropping substantially in open-ended scenarios. For non-RAG strategies, the primary failure mode was an internal knowledge deficit. The integration of RAG proved to be the most effective intervention. A domain-aligned RAG strategy not only significantly improved accuracy in complex decision-making tasks but also fundamentally shifted the system’s primary bottleneck from knowledge deficits to retrieval failures. In terms of safety, RAG induced a favorable shift from high-risk fabrications to safer refusals, though this introduced a safety-utility trade-off in the form of false refusals. Notably, our hybrid-RAG configuration, which combined both knowledge sources, demonstrated the most robust and generalizable performance across all tasks. Conclusions: Current LLMs lack the specialized knowledge required for reliable application in MSI oncology. A well-designed RAG architecture is the pivotal intervention to address this gap. However, its success is not automatic; it transforms the nature of system failure, making retrieval precision and knowledge base quality the new critical determinants of performance and safety. Our findings establish a clear directive for developing trustworthy clinical artificial intelligence: focus must shift toward optimizing the retrieval component and curating high-quality, comprehensive knowledge sources. MSIC-Bench provides a robust framework to guide these future efforts.
- 21 May 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e88614
Open AccessBenchmarking Large Language Models and Prompt Engineering Strategies in Microsatellite Instability Cancers: Evaluation Study
Background: The reliability of general-purpose large language models (LLMs) for complex clinical tasks in specialized domains such as microsatellite instability (MSI) cancers remains critically uncharacterized. The absence of a domain-specific benchmark to evaluate and guide the optimization of their capabilities across diverse clinical tasks poses unevaluated risks to patient safety. Objective: This study aimed to develop and validate Microsatellite Instability Cancer Benchmark (MSIC-Bench), a novel, two-tiered benchmark for MSI cancer, covering both consensus and frontier knowledge. Using this framework, we aimed to systematically assess LLM performance across various prompting strategies, identify task-specific weaknesses, and reveal effective pathways for performance improvement. Methods: We developed MSIC-Bench, a 511-question benchmark derived from clinical guidelines and a curated knowledge base. Three state-of-the-art LLMs (GPT-4o, [OpenAI], Gemini 2.5 Pro [Google], and Claude Opus 4 [Anthropic]) were evaluated using 4 prompting strategies, including vanilla, chain-of-thought, reflection of thoughts, and retrieval-augmented generation (RAG), under both multiple-choice and open-ended modalities. Performance was assessed on accuracy, safety (honesty), error composition, and token usage. Results: LLMs demonstrated a significant “scaffolding effect,” with accuracy dropping substantially in open-ended scenarios. For non-RAG strategies, the primary failure mode was an internal knowledge deficit. The integration of RAG proved to be the most effective intervention. A domain-aligned RAG strategy not only significantly improved accuracy in complex decision-making tasks but also fundamentally shifted the system’s primary bottleneck from knowledge deficits to retrieval failures. In terms of safety, RAG induced a favorable shift from high-risk fabrications to safer refusals, though this introduced a safety-utility trade-off in the form of false refusals. Notably, our hybrid-RAG configuration, which combined both knowledge sources, demonstrated the most robust and generalizable performance across all tasks. Conclusions: Current LLMs lack the specialized knowledge required for reliable application in MSI oncology. A well-designed RAG architecture is the pivotal intervention to address this gap. However, its success is not automatic; it transforms the nature of system failure, making retrieval precision and knowledge base quality the new critical determinants of performance and safety. Our findings establish a clear directive for developing trustworthy clinical artificial intelligence: focus must shift toward optimizing the retrieval component and curating high-quality, comprehensive knowledge sources. MSIC-Bench provides a robust framework to guide these future efforts.
- Open Access
- 4 April 2026
- Published by Elsevier in SSRN Electronic Journal
MTDP 1.0: An Evidence-Based Data Platform for Personalized Music Therapy
BackgroundThe advancement of AI music models (e.g., SUNO) and large language models (LLMs) such as GPT has created opportunities for personalized digital music therapy. However, the complex and diverse relationships between musical elements and diseases still lack systematic evidence support and personalized characterization.MethodsThe musical therapy data were collected from PubMed through systematic screening. A framework was developed using the PICOS model for manual extraction. LLM-powered automated extraction was integrated to construct the knowledge database. Dual verification was implemented for quality assurance, and dynamic charts and network relationship diagrams were visualized using the Vue3 framework and JavaScript libraries.ResultsBy systematically curating 328 music therapy publications and evidence from PubMed, we have developed a knowledge database that captures diverse clinical contexts, music intervention modalities, and parameters. This study establishes a comprehensive digital modeling framework consisting of four core components: Clinical Intervention Framework, Musical Content Framework, Social Delivery Framework, and Intervention Dosage Framework. The MTDP platform, its accompanying digital tools and dataset are publicly available at http://mtdp.bioinf.org.cn/ConclusionAs the first standardized and scalable knowledge database in this field, our platform bridges the gap between empirical diversity and digital precision, paving new avenues for research and application in personalized music-based health interventions.
- 4 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessMTDP 1.0: An Evidence-Based Data Platform for Personalized Music Therapy
BackgroundThe advancement of AI music models (e.g., SUNO) and large language models (LLMs) such as GPT has created opportunities for personalized digital music therapy. However, the complex and diverse relationships between musical elements and diseases still lack systematic evidence support and personalized characterization.MethodsThe musical therapy data were collected from PubMed through systematic screening. A framework was developed using the PICOS model for manual extraction. LLM-powered automated extraction was integrated to construct the knowledge database. Dual verification was implemented for quality assurance, and dynamic charts and network relationship diagrams were visualized using the Vue3 framework and JavaScript libraries.ResultsBy systematically curating 328 music therapy publications and evidence from PubMed, we have developed a knowledge database that captures diverse clinical contexts, music intervention modalities, and parameters. This study establishes a comprehensive digital modeling framework consisting of four core components: Clinical Intervention Framework, Musical Content Framework, Social Delivery Framework, and Intervention Dosage Framework. The MTDP platform, its accompanying digital tools and dataset are publicly available at http://mtdp.bioinf.org.cn/ConclusionAs the first standardized and scalable knowledge database in this field, our platform bridges the gap between empirical diversity and digital precision, paving new avenues for research and application in personalized music-based health interventions.
- Open Access
- 9 March 2026
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 14 (1) , e78519
Knowledge-Guided Explainable Recommendation Tool for Cancer Risk Prediction Models Using Retrieval-Augmented Large Language Models: Development and Validation Study
Background: Cancer risk prediction models are vital for precision prevention, enabling individualized assessment of cancer susceptibility based on genetic, clinical, environmental, and lifestyle factors. However, the practical use of these models is hindered by fragmented resources, heterogeneous reporting, and the absence of transparent, structured systems for systematic discovery and comparison. Objective: This study aimed to develop a retrieval-augmented, knowledge-guided system that provides accurate recommendations for cancer risk prediction models. Methods: We developed CanRisk-RAG, a recommendation platform underpinned by a precisely constructed knowledge base comprising more than 800 peer-reviewed cancer risk prediction models spanning diverse cancer types, modeling approaches, and predictive variables. The system integrates (1) large language model (LLM)–based semantic tag extraction, (2) embedding vectorization of structured metadata and abstracts, (3) a multifactor ranking algorithm combining semantic similarity with multiple quality indicators, and (4) LLM-generated literature summarization to support rapid user interpretation. Performance was evaluated across 4 types of representative queries. Eight domain experts independently assessed retrieval quality. CanRisk-RAG was benchmarked against PubMed, ChatGPT-4o, ScholarAI, and Gemini 1.5 Flash. Results: On the independent validation set, CanRisk-RAG consistently outperformed all 4 baseline applications, achieving the highest overall relevance (8.30 [SD 0.59]) and reliability (7.62 [SD 0.76]) scores on a 10-point scale ( P <.05). It also demonstrated high authenticity, data completeness, and consistency. Baseline applications frequently returned incomplete, inconsistent, or fabricated results, especially for complex, multifactorial queries, whereas CanRisk-RAG delivered accurate and structured recommendations grounded in validated evidence. Conclusions: CanRisk-RAG presents a transparent, domain-specific, and semantically enriched framework for discovering cancer risk prediction models, addressing several limitations of existing keyword-based search tools and general-purpose LLMs. By integrating structured knowledge, multifactor ranking, and LLM-based reasoning, the system aims to improve the precision, reproducibility, and usability of model selection in cancer risk prediction. While our evaluation demonstrates encouraging performance compared with baseline systems, further validation in broader clinical contexts and real-world applications is warranted. The framework’s general design may also be adaptable to other clinical model domains, providing a potential foundation for advancing evidence-based model discovery in precision medicine.
- 9 March 2026
- Published by JMIR Publications Inc. in JMIR Medical Informatics
- Vol. 14 (1) , e78519
Open AccessKnowledge-Guided Explainable Recommendation Tool for Cancer Risk Prediction Models Using Retrieval-Augmented Large Language Models: Development and Validation Study
Background: Cancer risk prediction models are vital for precision prevention, enabling individualized assessment of cancer susceptibility based on genetic, clinical, environmental, and lifestyle factors. However, the practical use of these models is hindered by fragmented resources, heterogeneous reporting, and the absence of transparent, structured systems for systematic discovery and comparison. Objective: This study aimed to develop a retrieval-augmented, knowledge-guided system that provides accurate recommendations for cancer risk prediction models. Methods: We developed CanRisk-RAG, a recommendation platform underpinned by a precisely constructed knowledge base comprising more than 800 peer-reviewed cancer risk prediction models spanning diverse cancer types, modeling approaches, and predictive variables. The system integrates (1) large language model (LLM)–based semantic tag extraction, (2) embedding vectorization of structured metadata and abstracts, (3) a multifactor ranking algorithm combining semantic similarity with multiple quality indicators, and (4) LLM-generated literature summarization to support rapid user interpretation. Performance was evaluated across 4 types of representative queries. Eight domain experts independently assessed retrieval quality. CanRisk-RAG was benchmarked against PubMed, ChatGPT-4o, ScholarAI, and Gemini 1.5 Flash. Results: On the independent validation set, CanRisk-RAG consistently outperformed all 4 baseline applications, achieving the highest overall relevance (8.30 [SD 0.59]) and reliability (7.62 [SD 0.76]) scores on a 10-point scale ( P <.05). It also demonstrated high authenticity, data completeness, and consistency. Baseline applications frequently returned incomplete, inconsistent, or fabricated results, especially for complex, multifactorial queries, whereas CanRisk-RAG delivered accurate and structured recommendations grounded in validated evidence. Conclusions: CanRisk-RAG presents a transparent, domain-specific, and semantically enriched framework for discovering cancer risk prediction models, addressing several limitations of existing keyword-based search tools and general-purpose LLMs. By integrating structured knowledge, multifactor ranking, and LLM-based reasoning, the system aims to improve the precision, reproducibility, and usability of model selection in cancer risk prediction. While our evaluation demonstrates encouraging performance compared with baseline systems, further validation in broader clinical contexts and real-world applications is warranted. The framework’s general design may also be adaptable to other clinical model domains, providing a potential foundation for advancing evidence-based model discovery in precision medicine.
- Open Access
- 1 March 2026
- Published by American Association for the Advancement of Science (AAAS) in Computational and Structural Biotechnology Journal
- Vol. 35 (1)
MSICKB: A Curated Knowledgebase for Exploring Molecular Heterogeneity and Biomarker Prioritization in Microsatellite Instability Cancers
Highlights: Curated MSI knowledgebase spanning 31 cancer types. MSICKB integrates 1,382 molecular and clinical features. A simple gene–cancer network identified 9 cross-cancer hub genes. Hub genes converged on mismatch repair and immune-related pathways. MSICKB supports cross-cancer comparison and biomarker prioritization.
- 1 March 2026
- Published by American Association for the Advancement of Science (AAAS) in Computational and Structural Biotechnology Journal
- Vol. 35 (1)
Open AccessMSICKB: A Curated Knowledgebase for Exploring Molecular Heterogeneity and Biomarker Prioritization in Microsatellite Instability Cancers
Highlights: Curated MSI knowledgebase spanning 31 cancer types. MSICKB integrates 1,382 molecular and clinical features. A simple gene–cancer network identified 9 cross-cancer hub genes. Hub genes converged on mismatch repair and immune-related pathways. MSICKB supports cross-cancer comparison and biomarker prioritization.
- 31 December 2025
- Published by American Chemical Society (ACS) in Journal of Chemical Information and Modeling
- Vol. 66 (1) , 122-137
DRPMKB1.0: A Comprehensive Knowledge Base for an AI-Oriented Drug Repositioning Prediction Model
Drug repositioning (DR) reduces the risks and costs of drug development by identifying new uses for approved drugs. The rapid growth of artificial intelligence (AI) has led to many computational models. However, without effective integration, excess models can waste resources and obscure valuable ones. While large language models (LLMs) are preferred for their broad applicability, integrating them with a personalized knowledge base improves task-specific accuracy. Thus, we developed the AI-oriented drug repositioning prediction model knowledge base (DRPMKB 1.0). This knowledge base compiles data from PubMed up to March 2024, covering two interfaces (display and interaction) and four dimensions (data, model, application, and reference). It includes 45 categories, 193 models, and 693 data entries, offering a comprehensive data sharing platform for DR. DRPMKB 1.0 establishes a dual-evaluation framework to standardize model selection, appraising both inherent model quality and the evidentiary support for its predictions. DRPMKB 1.0 integrates diverse data and models for personalized DR, offering tailored model recommendations based on user data, improving prediction accuracy. DRPMKB 1.0 also offers a foundation for developers to integrate models and data sets seamlessly. The knowledge base supports AI enhancement through a knowledge base for continuous model refinement.
- 31 December 2025
- Published by American Chemical Society (ACS) in Journal of Chemical Information and Modeling
- Vol. 66 (1) , 122-137
DRPMKB1.0: A Comprehensive Knowledge Base for an AI-Oriented Drug Repositioning Prediction Model
Drug repositioning (DR) reduces the risks and costs of drug development by identifying new uses for approved drugs. The rapid growth of artificial intelligence (AI) has led to many computational models. However, without effective integration, excess models can waste resources and obscure valuable ones. While large language models (LLMs) are preferred for their broad applicability, integrating them with a personalized knowledge base improves task-specific accuracy. Thus, we developed the AI-oriented drug repositioning prediction model knowledge base (DRPMKB 1.0). This knowledge base compiles data from PubMed up to March 2024, covering two interfaces (display and interaction) and four dimensions (data, model, application, and reference). It includes 45 categories, 193 models, and 693 data entries, offering a comprehensive data sharing platform for DR. DRPMKB 1.0 establishes a dual-evaluation framework to standardize model selection, appraising both inherent model quality and the evidentiary support for its predictions. DRPMKB 1.0 integrates diverse data and models for personalized DR, offering tailored model recommendations based on user data, improving prediction accuracy. DRPMKB 1.0 also offers a foundation for developers to integrate models and data sets seamlessly. The knowledge base supports AI enhancement through a knowledge base for continuous model refinement.
- Open Access
- 28 November 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Benchmarking LLMs and Prompt Engineering Strategies for Consensus and Frontier Knowledge in Microsatellite Instability Cancers (Preprint)
BACKGROUND The reliability of general-purpose Large Language Models (LLMs) for complex clinical tasks in specialized domains like microsatellite instability (MSI) cancers remains critically uncharacterized. The absence of a domain-specific benchmark to evaluate and guide the optimization of their capabilities across diverse clinical tasks poses unevaluated risks to patient safety. OBJECTIVE The primary objective was to develop and validate MSIC-Bench, a novel, two-tiered benchmark for MSI cancer covering both consensus and frontier knowledge. Using this framework, we aimed to systematically assess LLMs performance across various prompting strategies, identify task-specific weaknesses, and reveal effective pathways for performance improvement. METHODS We developed MSIC-Bench, a 500-question benchmark derived from clinical guidelines and a curated knowledge base. Three state-of-the-art LLMs (GPT-4o, Gemini 2.5 Pro, and Claude Opus 4) were evaluated using four prompting strategies, including vanilla, Chain-of-Thought (CoT), Reflection of Thoughts (RoT), and Retrieval-Augmented Generation (RAG), under both multiple-choice and open-ended modalities. Performance was assessed on accuracy, safety (honesty), error composition, and token usage. RESULTS A significant 'scaffolding effect' was observed, with the average LLMs accuracy dropping from 89.81% in multiple-choice formats to 76.56% in open-ended scenarios. Our task-specific analysis revealed this decline was most pronounced in complex therapeutic decision-making tasks. Error analysis attributed failures in non-RAG models primarily to insufficient domain knowledge (55.51% of errors), manifesting as a high frequency of unsafe fabrication. The integration of RAG proved highly effective, substantially improving accuracy in these critical tasks (e.g., boosting claude's performance from 76.8% to 90.4%) and inducing a crucial shift towards safety by increasing explicit statements of uncertainty (from 6.70% to 16.55% on average, and up to 75% in specific cases). Notably, these gains were achieved with significantly lower token usage (RAG: 115 tokens vs. CoT: 398 and RoT: 613 tokens on average for GPT-4o). CONCLUSIONS Our comprehensive evaluation reveals that LLMs lack the specialized domain knowledge required for complex MSI cancer-related tasks, rather than suffering from reasoning deficits. Prompting strategies substantially influence LLMs accuracy, safety, and token usage, with RAG emerging as the most effective and reliable method for improving both accuracy and safety. Ultimately, MSIC-Bench provides not only a comprehensive resource for systematic evaluation and optimization of LLMs in the MSI cancer domain, but its two-tiered design also offers a replicable blueprint for developing similar benchmarks in other knowledge-intensive medical fields.
- 28 November 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessBenchmarking LLMs and Prompt Engineering Strategies for Consensus and Frontier Knowledge in Microsatellite Instability Cancers (Preprint)
BACKGROUND The reliability of general-purpose Large Language Models (LLMs) for complex clinical tasks in specialized domains like microsatellite instability (MSI) cancers remains critically uncharacterized. The absence of a domain-specific benchmark to evaluate and guide the optimization of their capabilities across diverse clinical tasks poses unevaluated risks to patient safety. OBJECTIVE The primary objective was to develop and validate MSIC-Bench, a novel, two-tiered benchmark for MSI cancer covering both consensus and frontier knowledge. Using this framework, we aimed to systematically assess LLMs performance across various prompting strategies, identify task-specific weaknesses, and reveal effective pathways for performance improvement. METHODS We developed MSIC-Bench, a 500-question benchmark derived from clinical guidelines and a curated knowledge base. Three state-of-the-art LLMs (GPT-4o, Gemini 2.5 Pro, and Claude Opus 4) were evaluated using four prompting strategies, including vanilla, Chain-of-Thought (CoT), Reflection of Thoughts (RoT), and Retrieval-Augmented Generation (RAG), under both multiple-choice and open-ended modalities. Performance was assessed on accuracy, safety (honesty), error composition, and token usage. RESULTS A significant 'scaffolding effect' was observed, with the average LLMs accuracy dropping from 89.81% in multiple-choice formats to 76.56% in open-ended scenarios. Our task-specific analysis revealed this decline was most pronounced in complex therapeutic decision-making tasks. Error analysis attributed failures in non-RAG models primarily to insufficient domain knowledge (55.51% of errors), manifesting as a high frequency of unsafe fabrication. The integration of RAG proved highly effective, substantially improving accuracy in these critical tasks (e.g., boosting claude's performance from 76.8% to 90.4%) and inducing a crucial shift towards safety by increasing explicit statements of uncertainty (from 6.70% to 16.55% on average, and up to 75% in specific cases). Notably, these gains were achieved with significantly lower token usage (RAG: 115 tokens vs. CoT: 398 and RoT: 613 tokens on average for GPT-4o). CONCLUSIONS Our comprehensive evaluation reveals that LLMs lack the specialized domain knowledge required for complex MSI cancer-related tasks, rather than suffering from reasoning deficits. Prompting strategies substantially influence LLMs accuracy, safety, and token usage, with RAG emerging as the most effective and reliable method for improving both accuracy and safety. Ultimately, MSIC-Bench provides not only a comprehensive resource for systematic evaluation and optimization of LLMs in the MSI cancer domain, but its two-tiered design also offers a replicable blueprint for developing similar benchmarks in other knowledge-intensive medical fields.
- 3 April 2025
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 190, 110107
MIO: An ontology for annotating and integrating medical knowledge in myocardial infarction to enhance clinical decision making
- 3 April 2025
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 190, 110107
MIO: An ontology for annotating and integrating medical knowledge in myocardial infarction to enhance clinical decision making