Changyu Wang

Large language models (LLMs), such as OpenAI’s GPT series and Google’s PaLM, are transforming health care by improving clinical decision-making, enhancing patient communication, and simplifying administrative tasks. However, their performance relies heavily on prompt design, as small changes in wording or structure can greatly impact output quality. This presents challenges for clinicians who are not experts in natural language processing (NLP). This tutorial combines prompt engineering techniques tailored for clinical use, covering methods like zero-shot prompting, one-shot prompting, few-shot prompting, chain-of-thought prompting, self-consistency prompting, generated knowledge prompting, and meta-prompting. We provide actionable guidance on defining objectives, applying core principles, iterative prompt refinement, and integration into interoperable electronic health record (EHR) systems. This framework helps clinicians leverage LLMs to improve decision-making, streamline documentation, and enhance patient communication while maintaining ethical standards and ensuring patient safety.

Diagnostic errors in rare and complex diseases contribute significantly to morbidity and mortality. The ability of large language models (LLMs) to enhance diagnostic performance in such cases remains uncertain. This study compares the diagnostic accuracy, clinical reasoning quality, and inference efficiency of three ChatGPT variants (o3-mini, o3-mini-high, o1) and DeepSeek-R1 using 30 English-language case reports of rare and complex diseases from 26 specialties across 15 countries, sourced from PubMed and Web of Science Core Collection databases. Cases were selected to avoid overlap with model training data. Each case was processed once by each model, with outputs anonymized and evaluated in a double-blind manner by two board-certified physicians (each with >15 years’ clinical experience) and ChatGPT-4o. Diagnostic accuracy, the primary outcome, ranged between 30.0% and 40.0% with no significant differences observed among models (Cochran’s Q test, P = 0.16). ChatGPT-o1 achieved the highest accuracy (12/30, 40.0%; 95% CI, 24.6%+/-57.7%), followed by ChatGPT-o3-mini and o3-mini-high (each 11/30, 36.7%), and DeepSeek-R1 (9/30, 30.0% for each English and Chinese language inputs). Mean reasoning scores differed significantly (P < 0.05): ChatGPT-o1, 4.08 +/- 0.82; DeepSeek-R1 (English), 3.86 +/- 0.86; ChatGPT-o3-mini, 3.71 +/- 0.90; ChatGPT-o3-mini-high, 3.69 +/- 0.80; DeepSeek-R1 (Chinese), 3.67 +/- 0.84. Inter-evaluator agreement was high (ICC = 0.84; 95% CI, 0.80-0.88). Inference times varied significantly (P < 0.001), with ChatGPT-o3-mini being fastest (7.0 +/- 3.8 s) and DeepSeek-R1 (English) slowest (46.5 +/- 32.5 s). Advanced LLMs demonstrate potential to support diagnosis of rare and complex diseases, with transparent reasoning processes that may aid clinical decision-making and medical education. Further domain-specific refinement and prospective clinical validation are essential for safe and effective integration into clinical practice. Highlights: While LLMs showed similar diagnostic accuracy (30-40%) in rare and complex diseases, ChatGPT-o1 significantly excelled in the quality of its clinical reasoning. Inference speeds varied dramatically (7s-47s), highlighting a critical trade-off between model performance and real-world utility. The transparent reasoning of LLMs shows clear promise as a tool to support clinical decision-making and medical education. Safe clinical implementation is dependent on future domain-specific refinement and prospective validation.

UNSTRUCTURED Large language models (LLMs) present a promising avenue for improving healthcare by enhancing clinical decision making. However, their effectiveness heavily relies on the accurate prompt engineering. This review focuses on understanding and optimizing prompt engineering techniques to guide LLMs in producing clinically relevant and accurate responses. The aim is to provide clinicians with the tools to fully utilize LLMs in practice, ensuring patient-centered care and addressing ethical and operational challenges. Key principles of prompt engineering, such as specificity, contextual relevance and iterative refinement, are essential for the effective use of LLMs. Techniques such as zero-shot, few-shot and chain-of-thought prompting are analyzed in detail to provide clinicians with practical insights into how these approaches influence LLM outcomes. The review also introduces a classification system for prompts - manual versus automatic and discrete versus continuous - to help clinicians apply these models more effectively in different clinical scenarios. Despite advances, challenges remain in ensuring data privacy, maintaining clinical accuracy and handling multimodal data. Effective prompt engineering can significantly improve the performance of LLMs in clinical practice by optimizing input design to provide more accurate, contextually relevant and patient-specific outputs. These improvements enable more efficient clinical decision making. Clinicians need to consider privacy, ensure clinical accuracy and integrate adaptive, contextual and personalized prompts into real-time workflows. By refining prompt engineering practices, clinicians can take full advantage of LLM capabilities, ultimately improving patient outcomes and supporting ethical integration into healthcare.

Autoimmune inner ear disease (AIED) is a rare cause of sensorineural hearing loss, typically characterized by progressive bilateral hearing loss and vestibular symptoms. Diagnosing AIED is challenging due to the lack of specific biomarkers and its overlapping clinical features with other causes of hearing loss. This scoping review examines the recent advancements in the diagnostic approaches for AIED from 2020 to 2024, focusing on studies indexed in SCI, MEDLINE, and INSPEC. The review highlights progress in laboratory tests, imaging techniques, and genetic assessments, and discusses the implications of these advances in improving early diagnosis and treatment outcomes.

Background: Alzheimer’s disease (AD) is a progressive neurodegenerative disorder posing challenges to patients, caregivers, and society. Accessible and accurate information is crucial for effective AD management. Objective: This study aimed to evaluate the accuracy, comprehensibility, clarity, and usefulness of the Generative Pretrained Transformer’s (GPT) answers concerning the management and caregiving of patients with AD. Methods: In total, 14 questions related to the prevention, treatment, and care of AD were identified and posed to GPT-3.5 and GPT-4 in Chinese and English, respectively, and 4 respondent neurologists were asked to answer them. We generated 8 sets of responses (total 112) and randomly coded them in answer sheets. Next, 5 evaluator neurologists and 5 family members of patients were asked to rate the 112 responses using separate 5-point Likert scales. We evaluated the quality of the responses using a set of 8 questions rated on a 5-point Likert scale. To gauge comprehensibility and participant satisfaction, we included 3 questions dedicated to each aspect within the same set of 8 questions. Results: As of April 10, 2023, the 5 evaluator neurologists and 5 family members of patients with AD rated the 112 responses: GPT-3.5: n=28, 25%, responses; GPT-4: n=28, 25%, responses; respondent neurologists: 56 (50%) responses. The top 5 (4.5%) responses rated by evaluator neurologists had 4 (80%) GPT (GPT-3.5+GPT-4) responses and 1 (20%) respondent neurologist’s response. For the top 5 (4.5%) responses rated by patients’ family members, all but the third response were GPT responses. Based on the evaluation by neurologists, the neurologist-generated responses achieved a mean score of 3.9 (SD 0.7), while the GPT-generated responses scored significantly higher (mean 4.4, SD 0.6; P<.001). Language and model analyses revealed no significant differences in response quality between the GPT-3.5 and GPT-4 models (GPT-3.5: mean 4.3, SD 0.7; GPT-4: mean 4.4, SD 0.5; P=.51). However, English responses outperformed Chinese responses in terms of comprehensibility (Chinese responses: mean 4.1, SD 0.7; English responses: mean 4.6, SD 0.5; P=.005) and participant satisfaction (Chinese responses: mean 4.2, SD 0.8; English responses: mean 4.5, SD 0.5; P=.04). According to the evaluator neurologists’ review, Chinese responses had a mean score of 4.4 (SD 0.6), whereas English responses had a mean score of 4.5 (SD 0.5; P=.002). As for the family members of patients with AD, no significant differences were observed between GPT and neurologists, GPT-3.5 and GPT-4, or Chinese and English responses. Conclusions: GPT can provide patient education materials on AD for patients, their families and caregivers, nurses, and neurologists. This capability can contribute to the effective health care management of patients with AD, leading to enhanced patient outcomes.

Large language models (LLMs) represent a transformative advance in artificial intelligence, capable of understanding, generating and interpreting complex human language. Despite their generic design, these models have shown great potential in clinical applications, including medical diagnosis, clinical decision support, patient communication, and administrative tasks. However, deploying LLMs in clinical practice faces significant challenges, such as misinterpretation of human instructions, biased content generation, and the complexity of handling sensitive medical data. In this paper, we explore avenues for implementing LLMs in clinical settings, focusing on ethical considerations, bias mitigation, multimodal integration, simulation testing, clinical validation, and workflow integration. By addressing these challenges, we aim to improve the effectiveness, safety, and ethical standards of LLMs in healthcare, ensuring that they maximise benefits for both professionals and patients. The future outlook highlights the need for increased human-AI collaboration, improved model robustness, and adherence to ethical and regulatory standards. The ultimate goal is to provide comprehensive guidance on the effective use of LLM in clinical practice and to contribute to the development of reliable and ethical AI solutions in healthcare.

Background: Burnout among health care professionals is a significant concern, with detrimental effects on health care service quality and patient outcomes. The use of the electronic health record (EHR) system has been identified as a significant contributor to burnout among health care professionals. Objective: This systematic review and meta-analysis aims to assess the prevalence of burnout among health care professionals associated with the use of the EHR system, thereby providing evidence to improve health information systems and develop strategies to measure and mitigate burnout. Methods: We conducted a comprehensive search of the PubMed, Embase, and Web of Science databases for English-language peer-reviewed articles published between January 1, 2009, and December 31, 2022. Two independent reviewers applied inclusion and exclusion criteria, and study quality was assessed using the Joanna Briggs Institute checklist and the Newcastle-Ottawa Scale. Meta-analyses were performed using R (version 4.1.3; R Foundation for Statistical Computing), with EndNote X7 (Clarivate) for reference management. Results: The review included 32 cross-sectional studies and 5 case-control studies with a total of 66,556 participants, mainly physicians and registered nurses. The pooled prevalence of burnout among health care professionals in cross-sectional studies was 40.4% (95% CI 37.5%-43.2%). Case-control studies indicated a higher likelihood of burnout among health care professionals who spent more time on EHR-related tasks outside work (odds ratio 2.43, 95% CI 2.31-2.57). Conclusions: The findings highlight the association between the increased use of the EHR system and burnout among health care professionals. Potential solutions include optimizing EHR systems, implementing automated dictation or note-taking, employing scribes to reduce documentation burden, and leveraging artificial intelligence to enhance EHR system efficiency and reduce the risk of burnout. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42021281173; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021281173

Background: Artificial intelligence models tailored to diagnose cognitive impairment have shown excellent results. However, it is unclear whether large linguistic models can rival specialized models by text alone. Objective: In this study, we explored the performance of ChatGPT for primary screening of mild cognitive impairment (MCI) and standardized the design steps and components of the prompts. Methods: We gathered a total of 174 participants from the DementiaBank screening and classified 70% of them into the training set and 30% of them into the test set. Only text dialogues were kept. Sentences were cleaned using a macro code, followed by a manual check. The prompt consisted of 5 main parts, including character setting, scoring system setting, indicator setting, output setting, and explanatory information setting. Three dimensions of variables from published studies were included: vocabulary (ie, word frequency and word ratio, phrase frequency and phrase ratio, and lexical complexity), syntax and grammar (ie, syntactic complexity and grammatical components), and semantics (ie, semantic density and semantic coherence). We used R 4.3.0. for the analysis of variables and diagnostic indicators. Results: Three additional indicators related to the severity of MCI were incorporated into the final prompt for the model. These indicators were effective in discriminating between MCI and cognitively normal participants: tip-of-the-tongue phenomenon (P<.001), difficulty with complex ideas (P<.001), and memory issues (P<.001). The final GPT-4 model achieved a sensitivity of 0.8636, a specificity of 0.9487, and an area under the curve of 0.9062 on the training set; on the test set, the sensitivity, specificity, and area under the curve reached 0.7727, 0.8333, and 0.8030, respectively. Conclusions: ChatGPT was effective in the primary screening of participants with possible MCI. Improved standardization of prompts by clinicians would also improve the performance of the model. It is important to note that ChatGPT is not a substitute for a clinician making a diagnosis.

BACKGROUND Burnout among health care professionals is a significant concern, with detrimental effects on health care service quality and patient outcomes. The use of the electronic health record (EHR) system has been identified as a significant contributor to burnout among health care professionals. OBJECTIVE This systematic review and meta-analysis aims to assess the prevalence of burnout among health care professionals associated with the use of the EHR system, thereby providing evidence to improve health information systems and develop strategies to measure and mitigate burnout. METHODS We conducted a comprehensive search of the PubMed, Embase, and Web of Science databases for English-language peer-reviewed articles published between January 1, 2009, and December 31, 2022. Two independent reviewers applied inclusion and exclusion criteria, and study quality was assessed using the Joanna Briggs Institute checklist and the Newcastle-Ottawa Scale. Meta-analyses were performed using R (version 4.1.3; R Foundation for Statistical Computing), with EndNote X7 (Clarivate) for reference management. RESULTS The review included 32 cross-sectional studies and 5 case-control studies with a total of 66,556 participants, mainly physicians and registered nurses. The pooled prevalence of burnout among health care professionals in cross-sectional studies was 40.4% (95% CI 37.5%-43.2%). Case-control studies indicated a higher likelihood of burnout among health care professionals who spent more time on EHR-related tasks outside work (odds ratio 2.43, 95% CI 2.31-2.57). CONCLUSIONS The findings highlight the association between the increased use of the EHR system and burnout among health care professionals. Potential solutions include optimizing EHR systems, implementing automated dictation or note-taking, employing scribes to reduce documentation burden, and leveraging artificial intelligence to enhance EHR system efficiency and reduce the risk of burnout.

Background: Lung cancer patients with coronavirus disease 2019 (COVID-19) infection experience high mortality rates. The study aims to determine the risk factors for mortality in lung cancer patients with COVID-19 infection.Materials and methods: Followed the PRISMA reporting guidelines, PubMed, Embase, and Web of Science were systematically searched to February 20, 2023, for studies of lung cancer patients with COVID-19 infection. The main outcome of interest was the risk factor for mortality. We also compared the mortality rate of those patients among different continents. A pooled risk ratio (RR) with 95% CI was presented as the result of this meta-analysis.Results: Meta-analysis of 33 studies involving 5018 patients showed that pooled mortality rate of lung cancer in COVID-19 patients was 0.31 (95% CI: 0.25–0.36). Subgroup analysis based on the continents showed significant difference of the mortality rate was observed between Asia and the rest of world (χ²= 98.96,P< 0.01). Older age (SMD: 0.24, 95% CI: 0.09–0.40,P< 0.01), advanced lung cancer (RR: 1.14, 95% CI: 1.04–1.26,P< 0.01), coexisting comorbidities such as hypertension (RR: 1.17, 95% CI: 1.01–1.35,P= 0.04) and cardiovascular disease (RR: 1.40, 95% CI: 1.03–1.91,P= 0.03) were associated with higher risk of mortality rate in those patients.Conclusions: Findings of this meta-analysis confirms an increased risk of mortality in lung cancer patients with COVID-19 infection, whose risk factors for these patients appear to be exacerbated by older age, advanced-stage lung cancer, and comorbidities such as hypertension and cardiovascular disease.