Clinical prediction models in psychiatry: a systematic review of two decades of progress and challenges
Top Cited Papers
Open Access
- 1 April 2022
- journal article
- review article
- Published by Springer Nature in Molecular Psychiatry
- Vol. 27 (6) , 2700-2708
Abstract
Recent years have seen the rapid proliferation of clinical prediction models aiming to support risk stratification and individualized care within psychiatry. Despite growing interest, attempts to synthesize current evidence in the nascent field of precision psychiatry have remained scarce. This systematic review therefore sought to summarize progress towards clinical implementation of prediction modeling for psychiatric outcomes. We searched MEDLINE, PubMed, Embase, and PsychINFO databases from inception to September 30, 2020, for English-language articles that developed and/or validated multivariable models to predict (at an individual level) onset, course, or treatment response for non-organic psychiatric disorders (PROSPERO: CRD42020216530). Individual prediction models were evaluated based on three key criteria: (i) mitigation of bias and overfitting; (ii) generalizability, and (iii) clinical utility. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used to formally appraise each study’s risk of bias. 228 studies detailing 308 prediction models were ultimately eligible for inclusion. 94.5% of developed prediction models were deemed to be at high risk of bias, largely due to inadequate or inappropriate analytic decisions. Insufficient internal validation efforts (within the development sample) were also observed, while only one-fifth of models underwent external validation in an independent sample. Finally, our search identified just one published model whose potential utility in clinical practice was formally assessed. Our findings illustrated significant growth in precision psychiatry with promising progress towards real-world application. Nevertheless, these efforts have been inhibited by a preponderance of bias and overfitting, while the generalizability and clinical utility of many published models has yet to be formally established. Through improved methodological rigor during initial development, robust evaluations of reproducibility via independent validation, and evidence-based implementation frameworks, future research has the potential to generate risk prediction tools capable of enhancing clinical decision-making in psychiatric care.Keywords
Funding Information
- RCUK | Medical Research Council (MR/P005918/1)
- DH | National Institute for Health Research
- Wellcome Trust (202836/Z/16/Z)
This publication has 43 references indexed in Scilit:
- Single subject prediction of brain disorders in neuroimaging: Promises and pitfallsNeuroImage, 2017
- Adequate sample size for developing prediction models is not simply related to events per variableJournal of Clinical Epidemiology, 2016
- Detecting Neuroimaging Biomarkers for Psychiatric Disorders: Sample Size MattersFrontiers in Psychiatry, 2016
- Sample size considerations for the external validation of a multivariable prognostic model: a resampling studyStatistics in Medicine, 2015
- A new framework to enhance the interpretation of external validation studies of clinical prediction modelsJournal of Clinical Epidemiology, 2015
- Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpointsBMC Medical Research Methodology, 2014
- Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS ChecklistPLOS Medicine, 2014
- An empirical assessment of validation practices for molecular classifiersBriefings in Bioinformatics, 2011
- Age of onset of mental disorders: a review of recent literatureCurrent Opinion in Psychiatry, 2007
- Lifetime Prevalence and Age-of-Onset Distributions of DSM-IV Disorders in the National Comorbidity Survey ReplicationArchives of General Psychiatry, 2005