Use and Misuse of Waterfall Plots
Open Access
- 29 October 2014
- journal article
- research article
- Published by Oxford University Press (OUP) in JNCI: Journal of the National Cancer Institute
- Vol. 106 (12)
Abstract
“Waterfall plots” are used to describe changes in tumor size observed in clinical studies. Here we assess criteria for generation of waterfall plots and the impact of measurement error in generating them. We reviewed published waterfall plots to investigate variability in criteria used to define them. We then compared waterfall plots generated by different observers for 24 patients enrolled in a completed phase I study of solid tumors with available computed tomography (CT) scans. Tumor measurements were made independently from CT scans according to Response Evaluation Criteria in Solid Tumors 1.1 by four board-certified radiologists and four medical oncologists. Interobserver variability was quantified and compared with reference measurements reported for the phase 1 study. All statistical tests were two-sided. There was substantial variability in criteria used to generate published waterfall plots. In the internal study, the results were statistically significantly different between all eight readers (P = .01, variance = 197.1, SD = 14.0) and between the oncologists (P = .01, variance = 319.0, SD = 17.9), but not between the radiologists (P = .68, variance = 70.8, SD = 8.4). Different observers classified one to five patients as having a partial response and 12–19 patients as having stable disease. Similar variability in categorization of response was observed when these error rates were applied to published waterfall plots. Waterfall plots are subject to substantial variability in criteria used to define them and are influenced by measurement errors; they should be generated by trained radiologists. Caution should be exercised when interpreting results of waterfall plots in the context of clinical trials.Keywords
This publication has 13 references indexed in Scilit:
- A phase I trial of pantoprazole in combination with doxorubicin in patients with advanced solid tumors: evaluation of pharmacokinetics of both drugs and tissue penetration of doxorubicinInvestigational New Drugs, 2014
- Phase II Trial of Cetuximab With or Without Paclitaxel in Patients With Advanced Urothelial Tract CarcinomaJournal of Clinical Oncology, 2012
- Prospective Study of Bevacizumab Plus Temozolomide in Patients With Advanced Neuroendocrine TumorsJournal of Clinical Oncology, 2012
- Variability of Lung Tumor Measurements on Repeat Computed Tomography Scans Taken Within 15 MinutesJournal of Clinical Oncology, 2011
- New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1)European Journal of Cancer, 2009
- Variability in Mesothelioma Tumor Response ClassificationAmerican Journal of Roentgenology, 2006
- Measurement of mesothelioma on thoracic CT scans: A comparison of manual and computer‐assisted techniquesMedical Physics, 2004
- Interobserver and Intraobserver Variability in Measurement of Non–Small-Cell Carcinoma Lung Lesions: Implications for Assessment of Tumor ResponseJournal of Clinical Oncology, 2003
- Influence of measurement error on assessment of response to anticancer chemotherapy: proposal for new criteria of tumor response.Journal of Clinical Oncology, 1984
- The effect of measuring error on the results of therapeutic trials in advanced cancerCancer, 1976