Lesley A. Stewart

Objective: To explore editorial policies and experiences with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015 reporting guideline.Methods: Through a PubMed search, we identified the 25 journals that had published the highest number of systematic review protocols from May 2021 to May 2024. From corresponding journal websites, two authors independently extracted general and PRISMA-P specific reporting instructions and journal policies on publishing systematic review protocols. We distinguished between mandatory, non-mandatory, and generic terminology. We applied thematic content analysis to the extracted data. We emailed five editorial offices and 15 journal editors with invitations to participate in interviews on their experiences with PRISMA-P 2015. The interviews were conducted using a semi-structured interview guide around three prespecified themes: level of experience with publishing review protocols, views on using PRISMA-P 2015 in the editorial process, and reflections on its strengths and challenges. Framework analysis was applied to the interview transcripts.Results: From the 25 journals, we identified 143 descriptions of reporting instructions and journal policies. Sixty-two (43%, from 21 journals) were general reporting instructions, 27 (19%, from 11 journals) were PRISMA-P 2015 specific reporting instructions, and 54 (38%, from 19 journals) were journal policies on publishing systematic review protocols. Most (24 of 27, 89%) of the PRISMA-P 2015 reporting instructions were on use or endorsement (e.g., that authors were required or strongly encouraged to follow PRISMA-P 2015) or on requirements to submit a completed PRISMA-P 2015 checklist, frequently expressed using mandatory terminology. Ten journal editors were interviewed and several suggestions for modifying PRISMA-P 2015 emerged. Some suggestions related to the reporting guideline in general (e.g., add explanation on the distinction between registry entries and full protocols). Other suggestions related to adding or modifying content (e.g., report the current stage of the review or integrate guidance from other sources, such as PRISMA-Search).Conclusion: While nearly all of the 25 included journals had general reporting instructions and policies for publishing systematic review protocols, around half provided specific PRISMA-P 2015 reporting instructions. The interviewed journal editors suggested several additions and modifications to PRISMA-P 2015. Our findings provide context for editors, peer reviewers, authors, and other users of PRISMA-P 2015 and will inform the forthcoming update of the reporting guideline.

Background: Individual participant data (IPD) meta-analyses obtain, harmonise and synthesise the raw individual-level data from multiple studies, and are increasingly important in an era of data sharing and personalised medicine to inform clinical practice and policy. Objectives: (1) Describe the landscape of IPD meta-analysis of randomised trials over time; (2) establish current practice in design, conduct, analysis and reporting for pairwise IPD meta-analysis; and (3) derive recommendations to improve the conduct of and methods for future IPD meta-analyses. Design: Part 1: systematic review of all published IPD meta-analyses of randomised trials; Part 2: in-depth review of current methodological practice for pairwise IPD meta-analysis; and Part 3: adapted nominal group technique to derive consensus recommendations for IPD meta-analysis authors, educators and methodologists. Data sources: MEDLINE, Embase, and the Cochrane Database of Systematic Reviews (via the Ovid interface). Eligibility criteria: Part 1: all IPD meta-analyses of randomised trials published before February 2024, evaluating intervention effects and based on a systematic search. Part 2: all pairwise IPD meta-analyses from part 1 published between February 2022 and February 2024. Part 3: Selected panel of experienced IPD meta-analysis authors and/or methodologists. Results: Part 1: We identified 605 eligible IPD meta-analyses published between 1991 and 2024. The number of IPD meta-analyses published per year increased over time until 2019 but has since plateaued to about 60 per year. The most common clinical areas studied were cardiovascular disease (n=113, 19%) and cancer (n=110, 18%). The proportion of IPD meta-analyses published with Cochrane decreased over time from 16% (n=31/196) before 2015 to 3% (n=5/196) between 2021-2024. Part 2: 100 recent pairwise IPD meta-analyses were included in the in-depth review. Most cited PRISMA-IPD (68, 68%) and conducted risk of bias assessments (n=82, 82%), with just under half carrying out subgroup analyses not at risk of aggregation bias (n=36/85, 41%). However, only 33% (n=33) and 29% (n=29) respectively provided a protocol or statistical analysis plan, and only 7% (n=6/82) reported using IPD to inform risk of bias assessments. Part 3: 24 experts participated in a consensus workshop. Key recommendations for improved IPD meta-analyses focused on transparency (prospective registration; published protocols and statistical analysis plans) and maximising value (searching trial registries; obtaining IPD for unpublished evidence; using IPD to address missing data and risk of bias). Methodologists and educators should strengthen dissemination of methods and support capacity building across clinical fields and geographical areas. Conclusions: The application and methodological quality of IPD meta-analyses of randomised trials has increased in the last decade, but shortcomings remain. Implementing our consensus-based recommendations will ensure future IPD meta-analyses generate better evidence for clinical decision making. Study registration: Open Science Framework (1) Summary boxes: What is already known on this topic: IPD meta-analyses of randomised trials are regularly used to inform clinical policy and practice. They can provide better quality data and enable more thorough and robust analyses than standard aggregate data meta-analyses, but are resource-intensive and can be challenging to conduct, leading to variable methodological quality Previous studies that evaluated the conduct of IPD meta-analyses pre-date several major developments, such as the introduction of the PRISMA-IPD reporting guideline. What this study adds: This is the most comprehensive assessment of IPD meta-analyses of randomised trials to date (605 studies), showing an increase in publications over time followed by a recent plateau. The conduct of IPD meta-analysis has improved in recent years including increased use of prospective registration, assessment of risk of bias, appropriate analyses of patient subgroup effects and citing the PRISMA-IPD statement. Many shortcomings remain including (i) insufficient pre-specification of methods such as outcomes and analyses, (ii) sub-standard transparency (including publication of protocols, statistical analysis plans and reporting of analyses), and (iii) failure to gain maximum value of IPD (i.e. include unpublished trials, use the IPD to inform risk of bias and trustworthiness assessments, and address missing data appropriately); expert consensus recommendations are provided for how to address these gaps.

Objective: To explore systematic review protocol authors’ adherence to and experiences with the Preferred Reporting Items for Systematic reviews and Meta-analysis Protocols (PRISMA-P) 2015 reporting guideline.Methods: We randomly sampled 100 non-overlapping systematic review protocols from May 2021 to May 2024: 50 from PubMed-indexed journals and 50 from Open Science Framework (OSF)/The International Prospective Register of Systematic Reviews (PROSPERO). Two authors independently assessed adherence to the 26 PRISMA-P 2015 items as fully, partially (i.e., when some, but not all, aspects of an item were reported according to predefined criteria), or not reported (or not applicable). We analysed adherence within the two protocol types separately. Forty-three systematic review protocol authors were invited for interviews on their experiences with using PRISMA-P 2015. We used a semi-structured interview guide, involving three predefined themes: level of experience, views on using PRISMA-P 2015, and reflections on the guideline’s strengths and challenges. We applied framework analysis to the interview transcripts.Results: Among the 50 PubMed protocols, the median proportion of fully reported applicable PRISMA-P items was 60% (interquartile range 52%-64%). The median proportion for OSF/PROSPERO protocols was 45% (interquartile range 38%-58%). Across both types of protocols, >25% were found not to report six items: protocol amendments, role of funder, criteria for quantitative synthesis, methods for planned summary other than quantitative, meta-bias(es), and confidence in the cumulative evidence. In both types of protocols, >25% were found to partially report 11 items. Fifteen authors participated in the interviews. Through the interviews, suggestions for updating PRISMA-P 2015 emerged. Most suggestions concerned either adding or modifying existing content, e.g., to report conflicts of interest, or clarify guidance on how to report data synthesis when no meta-analysis was planned; other suggestions were more general, e.g., to add links to the Elaboration & Explanation paper. Conclusion: Adherence to PRISMA-P 2015 was low among 50 PubMed systematic review protocols, and even lower among 50 OSF/PROSPERO protocols. Six items were not reported across both types of protocols. The interviewed authors suggested various additions and modifications of the guideline. Findings from this study provide context for users of the reporting guideline and will inform a forthcoming update of PRISMA-P 2015.

Background: Diabetic retinopathy is a major cause of sight loss in people with diabetes, with a high risk of macular oedema, vitreous haemorrhage or other complications. Panretinal photocoagulation is the primary treatment for proliferative retinopathy. Anti-vascular endothelial growth factor drugs are used to treat various eye conditions and may be beneficial for people with proliferative or non-proliferative retinopathy. Methods: The Anti-VEGF In Diabetes project sought to investigate the clinical and cost-effectiveness of using anti-vascular endothelial growth factor to prevent retinopathy progression when compared to panretinal photocoagulation or no treatment. A systematic review with network meta-analysis of randomised controlled trials of anti-vascular endothelial growth factor (alone or in combination with panretinal photocoagulation) to treat retinopathy was conducted. The database searches were updated in May 2023. Individual participant data from larger trials were sought. A systematic review of non-randomised studies was performed. Existing cost-effectiveness analyses were reviewed, and a new economic model was developed, informed by the individual participant data meta-analysis. The model also estimated the value of undertaking further research to resolve decision uncertainty. Results: The review found that anti-vascular endothelial growth factors produced a slight, and not clinically meaningful, benefit over panretinal photocoagulation in best corrected visual acuity, after 1 year of follow-up in people with proliferative retinopathy (mean difference of 4.5 ETDRS letters; 95% credible interval −0.7 to 8.2). There was no evidence of a difference in effectiveness among the different anti-vascular endothelial growth factors. The benefit of anti-vascular endothelial growth factor appears to decline over time. Anti-vascular endothelial growth factor therapy may be more effective in people with poorer initial visual acuity. Anti-vascular endothelial growth factor had no impact on vision in people with non-proliferative retinopathy. Anti-vascular endothelial growth factor reduces rates of macular oedema and vitreous haemorrhage and may slow down the progression of retinopathy. Anti-vascular endothelial growth factors were predicted to be more costly but similarly effective to panretinal photocoagulation, with a net health benefit of −0.214 quality-adjusted life-years at a £20,000 willingness-to-pay threshold. Only under very select conditions might anti-vascular endothelial growth factors have the potential for cost-effectiveness to treat proliferative retinopathy. There is potentially significant value in reducing uncertainty through further primary research. Conclusions: Anti-vascular endothelial growth factor has no clinically meaningful benefit over panretinal photocoagulation for preserving visual acuity, but it may delay or prevent progression to macular oedema and vitreous haemorrhage. The long-term effectiveness and safety of anti-vascular endothelial growth factor treatment are unclear, particularly as additional panretinal photocoagulation and anti-vascular endothelial growth factor treatment will be required over time. Anti-vascular endothelial growth factors are therefore unlikely to be a cost-effective treatment for early proliferative retinopathy compared to panretinal photocoagulation. They are generally associated with higher costs and similar health outcomes across various scenarios. The long-term cost-effectiveness of anti-vascular endothelial growth factor is uncertain due to the lack of long-term clinical evidence. Future work: Further, robust studies with more than 2 years follow-up are required to evaluate the long-term efficacy and safety of anti-vascular endothelial growth factor use, and the effect of additional anti-vascular endothelial growth factor and panretinal photocoagulation therapy over time. Clinical trials or observational studies focusing on the use of anti-vascular endothelial growth factor in people with poorer vision at time of treatment may also be useful. Funding: This synopsis presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR132948.

Background: Proliferative diabetic retinopathy is a major cause of sight loss in people with diabetes, with a high risk of vitreous haemorrhage, tractional retinal detachment and other complications. Panretinal photocoagulation is the primary established treatment for proliferative diabetic retinopathy. Anti-vascular endothelial growth factor drugs are used to treat various eye conditions and may be beneficial for people with proliferative diabetic retinopathy. Objective: To investigate the efficacy and safety of anti-vascular endothelial growth factor therapy for the treatment of proliferative diabetic retinopathy when compared to panretinal photocoagulation. Methods: A systematic review and network meta-analysis of randomised controlled trials comparing anti-vascular endothelial growth factor (alone or in combination) to panretinal photocoagulation in people with proliferative diabetic retinopathy. The database searches were updated in May 2023. Trials where the primary focus was treatment of macular oedema or vitreous haemorrhage were excluded. Key outcomes were best corrected visual acuity, diabetic macular oedema and vitreous haemorrhage. Individual participant data were obtained and analysed for three large, high-quality trials in combination with published data from other trials. Network meta-analyses of best corrected visual acuity and meta-analyses of other outcomes combined individual participant data with published data from other trials; regression analyses against patient covariates used just the individual participant data. Results: Twelve trials were included: one of aflibercept, five of bevacizumab and six of ranibizumab. Individual participant data were available from 1 aflibercept and 2 ranibizumab trials, representing 624 patients (33% of the total). When considered together, anti-vascular endothelial growth factors produced a modest, but not clinically meaningful, benefit over panretinal photocoagulation in best corrected visual acuity, after 1 year of follow-up (mean difference in logarithm of the minimum angle of resolution −0.116, 95% credible interval −0.183 to −0.038). There was no clear evidence of a difference in effectiveness between the anti-vascular endothelial growth factors. The benefit of anti-vascular endothelial growth factor appears to decline over time. Analysis of the individual participant data trials suggested that anti-vascular endothelial growth factor therapy may be more effective in people with poorer visual acuity, in those who have vitreous haemorrhage and, possibly, in people with poorer vision generally. Anti-vascular endothelial growth factor was superior to panretinal photocoagulation at preventing macular oedema after 1 year (relative risk 0.48, 95% confidence interval 0.28 to 0.83) and possibly at preventing vitreous haemorrhage (relative risk 0.72, 95% confidence interval 0.47 to 1.10). Anti-vascular endothelial growth factor reduced the incidence of retinal detachment when compared to panretinal photocoagulation (relative risk 0.41, 95% confidence interval 0.22 to 0.77). Data on other adverse events were generally too limited to identify any differences between anti-vascular endothelial growth factor and panretinal photocoagulation. Conclusions: Anti-vascular endothelial growth factor has no clinically meaningful benefit over panretinal photocoagulation for preserving visual acuity. However, anti-vascular endothelial growth factor therapy appears to delay or prevent progression to macular oedema and vitreous haemorrhage. The possibility that anti-vascular endothelial growth factor therapy may be more effective in patients with poorer health and poorer vision merits further clinical investigation. The long-term effectiveness and safety of anti-vascular endothelial growth factor treatment are unclear, particularly as additional panretinal photocoagulation and anti-vascular endothelial growth factor treatment will be required over time. Funding: This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR132948.

This article consists of a citation of a published article describing research funded by the Health Technology Assessment programme under project number NIHR132948, and is provided as as part of the complete record of research outputs for this project. The original publication is available at: https://doi.org/10.1016/j.jval.2024.03.007 Objectives: This study aimed to evaluate the cost-effectiveness of anti–vascular endothelial growth factor drugs (anti-VEGFs) compared with panretinal photocoagulation (PRP) for treating proliferative diabetic retinopathy (PDR) in the United Kingdom. Methods: A discrete event simulation model was developed, informed by individual participant data meta-analysis. The model captures treatment effects on best corrected visual acuity in both eyes, and the occurrence of diabetic macular edema and vitreous hemorrhage. The model also estimates the value of undertaking further research to resolve decision uncertainty. Results: Anti-VEGFs are unlikely to generate clinically meaningful benefits over PRP. The model predicted anti-VEGFs be more costly and similarly effective as PRP, generating 0.029 fewer quality-adjusted life-years at an additional cost of £3688, with a net health benefit of −0.214 at a £20 000 willingness-to-pay threshold. Scenario analysis results suggest that only under very select conditions may anti-VEGFs offer potential for cost-effective treatment of PDR. The consequences of loss to follow-up were an important driver of model outcomes. Conclusions: Anti-VEGFs are unlikely to be a cost-effective treatment for early PDR compared with PRP. Anti-VEGFs are generally associated with higher costs and similar health outcomes across various scenarios. Although anti-VEGFs were associated with lower diabetic macular edema rates, the number of cases avoided is insufficient to offset the additional treatment costs. Key uncertainties relate to the long-term comparative effectiveness of anti-VEGFs, particularly considering the real-world rates and consequences of treatment nonadherence. Further research on long-term visual acuity and rates of vision-threatening complications may be beneficial in resolving uncertainties. Funding: This publication was funded by the Health Technology Assessment programme as a part of award number NIHR132948. This article reports on one component of the research award Anti-VEGF drugs compared with laser photocoagulation for the treatment of diabetic retinopathy: a systematic review and economic analysis. For more information about this research please view the award page [https://fundingawards.nihr.ac.uk/award/NIHR132948] DOI: https://doi.org/10.1016/j.jval.2024.03.007

Background: Non-proliferative and proliferative diabetic retinopathy are common complications of diabetes and a major cause of sight loss. Anti-vascular endothelial growth factor drugs represent a treatment option for people with diabetic retinopathy and are routinely used to treat various other eye conditions. However, anti-vascular endothelial growth factor drugs are expensive relative to current care options, and it is unclear whether this additional cost is justified when the immediate risk of vision loss is lower compared to patients with more aggressive ophthalmological conditions. Objective: To systematically review the evidence supporting the cost-effectiveness of alternative treatments for diabetic retinopathy. Methods: A systematic review of all comparative cost-effectiveness studies evaluating any treatment for diabetic retinopathy was conducted. Bibliographic searches were carried out to identify studies reporting on the cost-effectiveness of treatments for diabetic retinopathy; the latest searches were conducted on 28 April 2023. Included studies were synthesised narratively and evaluated with reference to UK decision-making. Studies were grouped by population into non-proliferative diabetic retinopathy and proliferative diabetic retinopathy. Results: The review identified five studies in the proliferative diabetic retinopathy population, all of which examined the cost-effectiveness of anti-vascular endothelial growth factor treatments compared to pan-retinal photocoagulation. Results of these studies suggest that anti-vascular endothelial growth factor treatments offer some additional benefits in terms of preserved visual acuity but also incur substantial additional costs relative to pan-retinal photocoagulation. Most authors agreed that the additional costs outweigh the limited benefits, especially in certain patient subgroups without pre-existing oedema. As most of the identified evidence considered a US perspective, it is unclear how these results would translate to a UK setting. Two studies were identified in the non-proliferative diabetic retinopathy population. There was limited evidence to support the early use of anti-vascular endothelial growth factor treatment. However, one UK study suggested that early treatment of non-proliferative diabetic retinopathy with pan-retinal photocoagulation is cost-effective compared to delayed pan-retinal photocoagulation. Conclusions: Overall, there is a dearth of cost-effectiveness evidence considering the UK context. The identified studies raised doubts about the cost-effectiveness of anti-vascular endothelial growth factor treatments for proliferative diabetic retinopathy. No conclusions can be made regarding the cost-effectiveness of anti-vascular endothelial growth factor treatments for non-proliferative diabetic retinopathy. Future research should focus on developing rigorous model-based cost-effectiveness analyses integrating all available evidence. Funding: This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR132948.

Background: Diabetic retinopathy is a major cause of sight loss in people with diabetes. The most severe form, proliferative diabetic retinopathy, carries a high risk of vision loss, vitreous haemorrhage, macular oedema and other harms. Panretinal photocoagulation is the primary treatment for proliferative diabetic retinopathy. Anti-vascular endothelial growth factor drugs are used to treat various eye conditions and may be beneficial for people with diabetic retinopathy. Objective: To investigate the efficacy and safety of anti-vascular endothelial growth factor therapy for the treatment of diabetic retinopathy when compared to panretinal photocoagulation. Methods: A systematic review and network meta-analysis of all published randomised controlled trials comparing anti-vascular endothelial growth factor (alone or in combination with panretinal photocoagulation) to panretinal photocoagulation in people with diabetic retinopathy. The database searches were updated in May 2023. Trials where the primary focus was treatment of macular oedema or vitreous haemorrhage were excluded. Results: A total of 14 trials were included: 3 of aflibercept, 5 of bevacizumab and 6 of ranibizumab. Two trials were of patients with non-proliferative diabetic retinopathy; all others were in proliferative diabetic retinopathy. Overall, anti-vascular endothelial growth factor was slightly better than panretinal photocoagulation at preventing vision loss, measured as best corrected visual acuity, at up to 2 years follow-up [mean difference in the logarithm of the minimum angle of resolution −0.089 (or 3.6 Early Treatment Diabetic Retinopathy Study letters), 95% confidence interval −0.180 to −0.019]. There was no clear evidence of any difference between the anti-vascular endothelial growth factors, but the potential for bias complicated the comparison. One trial found no benefit of anti-vascular endothelial growth factor over panretinal photocoagulation after 5 years. Anti-vascular endothelial growth factor was superior to panretinal photocoagulation at preventing macular oedema (relative risk 0.29, 95% confidence interval 0.18 to 0.49) and vitreous haemorrhage (relative risk 0.77, 95% confidence interval 0.61 to 0.99). There was no clear evidence that the effectiveness of anti-vascular endothelial growth factor varied over time. Conclusions: Anti-vascular endothelial growth factor injections reduce vision loss when compared to panretinal photocoagulation, but the benefit is small and unlikely to be clinically meaningful. Anti-vascular endothelial growth factor may have greater benefits for preventing complications such as macular oedema. Observational studies extending follow-up beyond the 1-year duration of most trials are needed to investigate the longer-term effects of repeated anti-vascular endothelial growth factor injections. Funding: This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR132948.

Objectives: To identify, summarise, and analyse published comments relevant to the PRISMA-P (Preferred Items for Reporting Systematic reviews and Meta-Analyses Protocols) 2015 reporting guideline for systematic review protocols, with special emphasis on suggestions for guideline modifications.Methods: We included documents (e.g., empirical studies and social media posts) that included comments relevant to PRISMA-P 2015. We searched bibliographic databases (e.g., Embase, MEDLINE, Scopus, from January 1st 2015 to February 2nd 2024) and other sources (e.g., BMJ rapid responses, BMC Blog Network, from January 1st 2015 to April 22nd 2024). Two authors independently assessed documents for inclusion, extracted data, and categorised comments. We categorised comments as ‘suggestion for modification to the wording of an existing PRISMA-P 2015 item’, ‘suggestion for a new item’, ‘suggestion for deletion of an existing PRISMA-P 2015 item’, or ‘additional comment’. We categorised each comment into themes and provided a summary and examples of the proposed suggestions. We analysed the characteristics of the suggestions by describing the rationale and comparing with existing PRISMA-P 2015 guidance.Results: We assessed full text of 1,912 potentially eligible documents and included 28 documents with 38 comments. Eleven comments suggested modifications to existing guideline items. Multiple comments proposed modifications to items related to eligibility criteria (three comments made different suggestions, e.g., one comment suggested to include reporting guidance relating to retracted papers in PRISMA 2020 and we deemed this potentially relevant for systematic review protocols) and data synthesis (three comments made different suggestions, e.g., one comment suggested to add reporting guidance relation to prediction intervals for random-effects meta-analyses in PRISMA 2020 and we deemed this potentially relevant for systematic review protocols). There were 11 comments suggesting new items. The data items section of PRISMA-P 2015 received the most comments (five comments made different suggestions, e.g., three comments suggested to add content on pre-specifying whether authors plan to extract information on funding and conflicts of interest among the included studies). Most of the suggestions provided a rationale directly in the document and around two-thirds of the suggestions addressed content not already included in PRISMA-P 2015.Conclusion: The issues raised provide context to authors, peer reviewers, editors, and readers of systematic review protocols using PRISMA-P 2015 and inform the planned update of the guideline.