Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability
Authors:
- Andrija BabicABAndrija Babic
- Ruzica TokalicRTRuzica Tokalic
- João Amílcar Silva CunhaJCJoão Amílcar Silva Cunha
- Ivana NovakINIvana Novak
- Jelena SutoJSJelena Suto
- Marin VidakMVMarin Vidak
- Ivana MiosicIMIvana Miosic
- Ivana VukaIVIvana Vuka
- Tina Poklepovic PericicTina Poklepovic Pericic
- Livia PuljakLPLivia Puljak
Open Access
- 5 April 2019
- journal article
- review article
- Published by Springer Nature in BMC Medical Research Methodology
- Vol. 19 (1) , 1-10
Abstract
An important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in Cochrane reviews of interventions published in the Cochrane Database of Systematic Reviews (CDSR). We analyzed Cochrane reviews published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many Cochrane reviews had different judgments for the same supporting explanations. In the main analysis we included 10,292 judgments and supporting explanations for attrition bias from 729 Cochrane reviews. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of Cochrane review authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same Cochrane review. We found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochrane’s risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare.Keywords
This publication has 13 references indexed in Scilit:
- Judgments of other bias in Cochrane systematic reviews of interventions are highly inconsistent and thus hindering use and comparability of evidencebioRxiv, 2018
- Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective studySystematic Reviews, 2017
- Handling missing data in RCTs; a review of the top medical journalsBMC Medical Research Methodology, 2014
- Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairsJournal of Clinical Epidemiology, 2013
- PEDro's bias: summary quality scores should not be used in meta-analysisJournal of Clinical Epidemiology, 2013
- Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic reviewBMJ, 2012
- Evidence-Based Medicine and The Cochrane Collaboration2009
- Reporting attrition in randomised controlled trialsBMJ, 2006
- Bias in Clinical Intervention ResearchAmerican Journal of Epidemiology, 2006
- What is meant by intention to treat analysis? Survey of published randomised controlled trialsBMJ, 1999