Development and internal validation of a diagnostic prediction model for life-threatening events in callers with shortness of breath: a cross-sectional study in out-of-hours primary care
Authors:
- Michelle SpekMSMichelle Spek
- Roderick P. VenekampRVRoderick P. Venekamp
- Anne A.H. de HondAHAnne A.H. de Hond
- Esther de GrootEGEsther de Groot
- Geert-Jan GeersingGGGeert-Jan Geersing
- Anna Sm DobbeADAnna Sm Dobbe
- Mathé DelissenMDMathé Delissen
- Frans H RuttenFRFrans H Rutten
- Maarten van SmedenMSMaarten van Smeden
- Dorien L ZwartDZDorien L Zwart
Open Access
- 21 January 2025
- journal article
- Published by Royal College of General Practitioners in British Journal of General Practice
- Vol. 75 (756) , e500-e508
Abstract
Background: Shortness of breath often prompts calls to out-of-hours primary care and is the prime reason for home visits by GPs. However, a diagnostic prediction model in this setting is lacking. Aim: To develop and internally validate a model predicting life-threatening events for out-of-hours primary care callers with shortness of breath. Design and setting: This cross-sectional study includes data from 1952 patients with shortness of breath who called Dutch out-of-hours primary care between September 2020 and August 2021. The trial was registered with the Netherlands Trial Register, number: NL9682. Method: Four logistic regression models were developed with life-threatening events as the outcome. The first model included age and gender (model 1) and then, successively, call characteristics (calling at night and someone else calling on behalf of the patient; model 2), symptoms (cough, fever, inability to speak full sentences and wheezing; model 3), and medical history and medication use (cardiovascular and/or pulmonary; model 4) were added. The models were internally validated using optimism correction via bootstrap with 1000 repetitions. Performance measures of discrimination (c-statistic) and calibration (calibration intercept and slope) were determined. Results: Approximately 17% (329/1952) of callers with shortness of breath had a life-threatening event. Model 3 performed best. This model exhibited good discriminative ability (internal validation c-statistic of 0.76, 95% confidence interval = 0.74 to 0.79) and was well calibrated. All models had a high net benefit compared with using no model. Models 3 and 4 had a higher net benefit compared with models 1 and 2. As models 3 and 4 were similar in terms of net benefit, the model with fewer parameters (model 3) is preferred. Conclusion: A prediction model consisting of age, gender, call characteristics, and symptoms holds promise for improving telephone triage of callers to out-of-hours primary care with shortness of breath. However, as the study period coincided with the COVID-19 pandemic, temporal validation is necessary before widespread implementation.Keywords
This publication has 39 references indexed in Scilit:
- Multimarker Strategy for Short-Term Risk Assessment in Patients With Dyspnea in the Emergency DepartmentJACC, 2012
- Likelihood of acute coronary syndrome in emergency department chest pain patients varies with time of presentationBMC Research Notes, 2012
- The appropriateness of, and compliance with, telephone triage decisions: a systematic review and narrative synthesisJournal of Advanced Nursing, 2012
- MissForest—non-parametric missing value imputation for mixed-type dataBioinformatics, 2011
- Validity of telephone and physical triage in emergency care: The Netherlands Triage SystemFamily Practice, 2010
- Verbal dyspnoea score predicts emergency department departure status in patients with shortness of breathEmergency Medicine Australasia, 2010
- Multimarker strategy for risk prediction in patients presenting with acute dyspnea to the emergency departmentInternational Journal of Cardiology, 2008
- Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency?Quality and Safety in Health Care, 2007
- Decision Curve Analysis: A Novel Method for Evaluating Prediction ModelsMedical Decision Making, 2006
- Decision making by emergency nurses in triage assessmentsAccident and Emergency Nursing, 1998