Comparative analysis of three risk assessment tools in Australian patients with prostate cancer
- 15 November 2011
- journal article
- research article
- Published by Wiley in BJU International
- Vol. 108 (s2) , 51-56
Abstract
What's known on the subject? and What does the study add? Prognostic tools, such as the Cancer of the Prostate Risk Assessment (CAPRA) score and the 1998 Kattan and 2006 Stephenson nomograms, predicting biochemical recurrence after radical prostatectomy are widely used for treatment decision making and counselling patients. However, tools derived in certain cohorts tend to perform less well when they are applied to populations that are dissimilar in terms of population or disease characteristics, health systems or treatment practices. Some of the loss in accuracy of a prognostic tool is a consequence of unknown factors and hence the performance of a tool when applied to a different population is unknown and largely unpredictable. This study validates these widely used tools in South Australian patients treated at three public hospitals. All three tools discriminated well according to risk of recurrence in these patients. However, when compared against observed rates of recurrence, it was found that predictions of recurrence varied widely between the three tools, suggesting that their use in counselling patients on such risk may not be appropriate. Interestingly, the oldest of the three tools (Kattan 1998) was the best predictor of absolute risk of recurrence. In the paper, this is linked to later adoption of updated Gleason grading, among other factors. SUMMARY: In many countries, prognostic tools, which draw on the experience of thousands of patients with cancer, are used to predict cancer outcomes, but accuracy varies. This paper compares the accuracy of three widely used tools predicting prostate cancer recurrence after surgery in Australian patients. The results show that all tools were good at predicting which patients were most likely to experience recurrence and which were least. However, prediction of absolute risk varied and the oldest tool was the most accurate. OBJECTIVE: To compare performance of the CAPRA score and two commonly used risk assessment nomograms, the 1998 Kattan and the 2006 Stephenson, in an untested Australian cohort. PATIENTS AND METHODS: We present data on 635 men from the South Australian Prostate Cancer Clinical Outcomes Database who underwent radical prostatectomy between January 1996 and May 2009 and had all required variables for predicting biochemical recurrence (BCR). BCR was defined as prostate‐specific antigen ≥0.2 ng/mL or secondary treatment for a rising prostate‐specific antigen. Accuracy was evaluated using Harrell's concordance index, plotting calibration curves, and constructing decision analysis curves. RESULTS: Concordance indices were high for all three tools: 0.791, 0.787 and 0.744 for the 2006 Stephenson nomogram, CAPRA score and 1998 Kattan nomogram respectively. At 3 years, calibration of the tools (agreement between predicted and observed BCR‐free probability) was close to ideal for the 1998 Kattan nomogram, whereas the 2006 Stephenson model underestimated and the CAPRA model overestimated BCR‐free probability. The 1998 Kattan and 2005 CAPRA tools performed better than the 2006 Stephenson nomogram across a wide range of threshold probabilities using decision curve analysis. CONCLUSION: All three tools discriminate between patients' risk effectively. Absolute estimates of risk are likely to vary widely between tools, however, suggesting that models should be validated and, if necessary, recalibrated in the population to which they will be applied. Recent development does not mean a nomogram is more accurate for use in a particular population.Keywords
This publication has 28 references indexed in Scilit:
- Clinical and socio-demographic profile of an Australian multi-institutional prostate cancer cohortAsia-Pacific Journal of Clinical Oncology, 2009
- Prostate cancer risk assessmentCancer, 2008
- Surgeon Experience is Strongly Associated With Biochemical Recurrence After Radical Prostatectomy for All Preoperative Risk CategoriesJournal of Urology, 2008
- Validity of the CAPRA Score to Predict Biochemical Recurrence-Free Survival After Radical Prostatectomy. Results From a European Multicenter Survey of 1,296 PatientsJournal of Urology, 2007
- The Surgical Learning Curve for Prostate Cancer Control After Radical ProstatectomyJNCI: Journal of the National Cancer Institute, 2007
- Decision Curve Analysis: A Novel Method for Evaluating Prediction ModelsMedical Decision Making, 2006
- Defining Biochemical Recurrence of Prostate Cancer After Radical Prostatectomy: A Proposal for a Standardized DefinitionJournal of Clinical Oncology, 2006
- Preoperative Nomogram Predicting the 10-Year Probability of Prostate Cancer Recurrence After Radical ProstatectomyJNCI: Journal of the National Cancer Institute, 2006
- Nomograms for prostate cancer—is their use evidence-based?Nature Reviews Endocrinology, 2005
- THE UNIVERSITY OF CALIFORNIA, SAN FRANCISCO CANCER OF THE PROSTATE RISK ASSESSMENT SCORE: A STRAIGHTFORWARD AND RELIABLE PREOPERATIVE PREDICTOR OF DISEASE RECURRENCE AFTER RADICAL PROSTATECTOMYJournal of Urology, 2005