Disagreements in risk of bias assessment for randomized controlled trials in hypertension-related Cochrane reviews
Open Access
- 21 June 2024
- journal article
- research article
- Published by Springer Nature in Trials
- Vol. 25 (1) , 1-13
Abstract
Background: The inter-reviewer reliability of the risk of bias (RoB) assessment lacked agreement in previous studies. It is important to analyse these disagreements to improve the repeatability of RoB assessment. The objective of the study was to evaluate the frequency and reasons for disagreements in RoB assessments for randomised controlled trials (RCTs) that were included in multiple Cochrane reviews in the field of hypertension. Methods: A cross-sectional study was employed. We retrieved any RCTs that had been included in multiple Cochrane reviews in the field of hypertension from ARCHIE. The results of the RoB assessments were extracted, and the distributions of agreements and possible reasons for disagreement were analyzed. Results: Twenty-six Cochrane reviews were included in this study. A total of 78 RCTs appeared in more than one Cochrane review. The level of agreement ranged from domain to domain. “Blinding of outcome assessment” showed a reasonably high level of agreement (94.9%), while “incomplete outcome data”, “selective outcome reporting” and “other sources of bias” showed moderate levels of agreement (74.6%, 79.2% and 75.6%, respectively). However, the domains of “allocation concealment”, “random sequence generation” and “blinding of participants and personnel” showed low levels of agreement (24.4%, 23.5%, and 47.4%, respectively). In the domains of “allocation concealment” and “blinding of participants and personnel”, the agreement group had higher proportion of publication year ≤ 1996 than the disagreement group (P = 0.008 and P < 0.001, respectively). In the “blinding of participants and personnel”, the impact factor was higher in the agreement group (P < 0.001). By analyzing the support text, we found that the most likely reason for disagreement was extracting different information from the same RCT. Conclusion: For Cochrane reviews in the field of hypertension using the 2011 version of the RoB tool, there was a large disagreement in the RoB assessment. It is suggested that the results of RoB assessments in systematic reviews that used the 2011 version of the RoB tool need to be interpreted with caution. More accurate information from RCTs needs to be collected when we synthesize clinical evidence.Keywords
Funding Information
- Sichuan Provincial Science and Technology Support Program (2023YFS0027, 2023YFQ0071)
- National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University (Z20191009)
This publication has 27 references indexed in Scilit:
- There were large discrepancies in risk of bias tool judgments when a randomized controlled trial appeared in more than one systematic reviewJournal of Clinical Epidemiology, 2016
- A systematic review of orthopaedic manual therapy randomized clinical trials qualityJournal of Manual & Manipulative Therapy, 2016
- Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairsJournal of Clinical Epidemiology, 2013
- CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trialsBMJ, 2010
- Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological studyBMJ, 2008
- Medical Journals Are an Extension of the Marketing Arm of Pharmaceutical CompaniesPLOS Medicine, 2005
- Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta‐epidemiological’ researchStatistics in Medicine, 2002
- Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?The Lancet, 1998
- Systematic Reviews: Synthesis of Best Evidence for Clinical DecisionsAnnals of Internal Medicine, 1997
- Improving the quality of reporting of randomized controlled trials. The CONSORT statementJAMA, 1996