Replicating the Violence Risk Appraisal Guide: A Total Forensic Cohort Study
Authors:
Open Access
- 14 March 2014
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 9 (3) , e91845
Abstract
The performance of violence risk assessment instruments can be primarily investigated by analysing two psychometric properties: discrimination and calibration. Although many studies have examined the discrimination capacity of the Violence Risk Appraisal Guide (VRAG) and other actuarial risk assessment tools, few have evaluated how well calibrated these instruments are. The aim of the present investigation was to replicate the development study of the VRAG in Europe including measurements of discrimination and calibration. Using a prospective study design, we assessed a total cohort of violent offenders in the Zurich Canton of Switzerland using the VRAG prior to discharge from prisons, secure facilities, and outpatient clinics. Assessors adhered strictly to the assessment protocol set out in the instrument’s manual. After controlling for attrition, 206 offenders were followed in the community for a fixed period of 7 years. We used charges and convictions for subsequent violent offenses as the outcomes. Receiver operating characteristic analysis was conducted to measure discrimination, and Sanders’ decomposition of the Brier score as well as Bayesian credible intervals were calculated to measure calibration. The discrimination of the VRAG’s risk bins was modest (area under the curve = 0.72, 95% CI = 0.63–0.81, p<0.05). However, the calibration of the tool was poor, with Sanders’ calibration score suggesting an average assessment error of 21% in the probabilistic estimates associated with each bin. The Bayesian credible intervals revealed that in five out of nine risk bins the intervals did not contain the expected risk rates. Measurement of the calibration validity of risk assessment instruments needs to be improved, as has been done with respect to discrimination. Additional replication studies that focus on the calibration of actuarial risk assessment instruments are needed. Meanwhile, we recommend caution when using the VRAG probabilistic risk estimates in practice.Keywords
This publication has 40 references indexed in Scilit:
- Predictive Validity Performance Indicators in Violence Risk Assessment: A Methodological PrimerBehavioral Sciences & the Law, 2013
- Current Obstacles in Replicating Risk Assessment Findings: A Systematic Review of Commonly Used Actuarial InstrumentsBehavioral Sciences & the Law, 2013
- Absolute Recidivism Rates Predicted By Static-99R and Static-2002R Sex Offender Risk Assessment Tools Vary Across SamplesCriminal Justice and Behavior, 2012
- Predictive and incremental validity of the Violence Risk Appraisal Guide scores with male and female jail inmates.Psychological Assessment, 2011
- Assessing the Performance of Prediction ModelsEpidemiology, 2010
- A Mathematical Proof and Example That Bayes's Theorem Is Fundamental to Actuarial Estimates of Sexual Recidivism RiskSexual Abuse, 2008
- Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r.Law and Human Behavior, 2005
- The Measurement of Interrater Agreement2003
- A Multisite Comparison of Actuarial Risk Instruments for Sex Offenders.Psychological Assessment, 2003
- Probabilistic prediction in patient management and clinical trialsStatistics in Medicine, 1986