Evaluation of New Treatments in Radiation Oncology
Authors:
- Ambuj KumarAKAmbuj Kumar
- Stephanie DanielsSDStephanie Daniels
- Suzanne SwannSSSuzanne Swann
- Alan CantorACAlan Cantor
- Iztok HozoIHIztok Hozo
- Mike ClarkMCMike Clark
- Fadila SerdarevicFSFadila Serdarevic
- Clement GwedeCGClement Gwede
- Andy TrottiATAndy Trotti
- Benjamin DjulbegovicBDBenjamin Djulbegovic
- 23 February 2005
- journal article
- research article
- Published by American Medical Association (AMA) in JAMA
- Vol. 293 (8) , 970-8
Abstract
The superiority of innovative over standard treatments is not known. To describe accurately the outcomes of innovations that are tested in randomized controlled trials (RCTs) 3 factors have to be considered: publication rate, quality of trials, and the choice of the adequate comparator intervention. To determine the success rate of innovative treatments by assessing preferences between experimental and standard treatments according to original investigators' conclusions, determining the proportion of RCTs that achieved primary outcomes' statistical significance, and performing meta-analysis to examine if the summary point estimate favored innovative vs standard treatments. Randomized controlled trials conducted by the Radiation Therapy Oncology Group (RTOG). All completed phase 3 trials conducted by the RTOG since its creation in 1968 until 2002. For multiple publications of the same study, we used the one with the most complete primary outcomes and with the longest follow-up information. We used the US National Cancer Institute definition of completed studies to determine the publication rate. We extracted data related to publication status, methodological quality, and treatment comparisons. One investigator extracted the data from all studies and 2 independent investigators extracted randomly about 50% of the data. Disagreements were resolved by consensus during a meeting. Data on 12,734 patients from 57 trials were evaluated. The publication rate was 95%. The quality of trials was high. We found no evidence of inappropriateness of the choice of comparator. Although the investigators judged that standard treatments were preferred in 71% of the comparisons, when data were meta-analyzed innovations were as likely as standard treatments to be successful (odds ratio for survival, 1.01; 99% confidence interval, 0.96-1.07; P = .5). In contrast, treatment-related mortality was worse with innovations (odds ratio, 1.76; 99% confidence interval, 1.01-3.07; P = .008). We found no predictable pattern of treatment successes in oncology: sometimes innovative treatments are better than the standard ones and vice versa; in most cases there were no substantive differences between experimental and conventional treatments. The finding that the results in individual trials cannot be predicted in advance indicates that the system and rationale for RCTs is well preserved and that successful interventions can only be identified after an RCT is completed.Keywords
This publication has 40 references indexed in Scilit:
- Absence of evidence is not evidence of absenceBMJ, 2004
- Well informed uncertainties about the effects of treatmentsBMJ, 2004
- Effect of interpretive bias on research evidenceBMJ, 2003
- Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trialsJAMA, 1995
- Cumulative meta-analysis of clinical trials builds evidence for exemplary medical careJournal of Clinical Epidemiology, 1995
- Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boardsJAMA, 1992
- Comparing benefits and harms: the balance sheet1990
- Measuring Gain in the Evaluation of Medical Technology The Probability of a Better OutcomeInternational Journal of Technology Assessment in Health Care, 1988
- Equipoise and the Ethics of Clinical ResearchNew England Journal of Medicine, 1987
- Statistics and Ethics in Surgery and AnesthesiaScience, 1977