Bethany Shinkins

Background: Cancer-specific mortality traditionally has been used to evaluate the effectiveness of cancer screening, and reduction of late-stage disease has been found to correlate with reduction of cancer-specific mortality, but measurement requires long follow-up. Increased detection of early-stage cancer has been proposed as a potential early indicator of beneficial interventions. Our systematic review and meta-analysis evaluated trial-level correlations between early-stage outcomes (early-stage cancer incidence, proportion of screen-detected cancers, diagnostic yield of screening) and cancer-specific mortality from all available cancer screening randomised controlled trials (RCTs). Methods: Weighted Pearson correlation coefficients were estimated for early-stage diagnosis outcomes (log relative risk for early-stage cancer incidence; proportion otherwise) and cancer-specific mortality. Results: We included 213 publications from 58 trials (62 trial-arm comparisons) covering 10 cancer types, but only 15 trials reported cancer stage at a timepoint prior to mortality. The correlation between early-stage cancer incidence and cancer mortality was −0.25 (95% CI: −0.49 to 0.07; 43 trials). This was similar by cancer type (e.g., lung [ n = 14]: −0.36 [95% CI: −0.69 to 0.15]; breast [ n = 13]: −0.21 [95% CI: −0.61 to 0.42]). The correlation for the proportion of screen-detected cancers was −0.16 (95% CI: −0.50 to 0.29; 44 trials). Correlation between the diagnostic yield and cancer mortality was −0.02 (95% CI: −0.72 to 0.45) for bowel ( n = 11) and −0.32 (95% CI: −0.83 to 0.19) for breast cancer ( n = 11). Conclusions: For cancer screening RCTs, neither early-stage cancer incidence, the proportion of screen-detected cancers nor diagnostic yield of screening demonstrated a reliable correlation with cancer mortality.

Objectives: To undertake an early economic evaluation of anti-cyclic citrullinated peptide (anti-CCP) antibody testing scenarios in primary care, to improve identification of individuals at risk of rheumatoid arthritis (RA). Methods: A discrete event simulation (DES) model was developed to track a hypothetical cohort of 10 000 patients from primary care presentation with musculoskeletal (MSK) symptoms without synovitis through to secondary care rheumatology services. Multiple anti-CCP testing scenarios were evaluated, including testing all patients on first presentation and targeted testing using a novel anti-CCP prediction model. Patients with positive anti-CCP results were referred to rheumatology services. The base-case assumed no primary care anti-CCP testing with no immediate referrals. Cost-effectiveness was assessed according to the additional cost to the healthcare system required to immediately refer one patient with imminent inflammatory arthritis (IA) (developing RA or other IA within 1 year of presentation). Results: Anti-CCP testing in all patients with MSK symptoms leads to 52.7% of those with imminent IA being immediately referred, versus 23.8% with targeted testing. Average per-patient short-term healthcare costs were £77.44 (no testing), £127.83 (testing all) and £87.54 (targeted testing). Versus no testing, additional costs of £5038 (testing all) or £2234 (targeted testing) were incurred per additional patient with imminent IA immediately referred. Targeted testing incurred more favorable cost-per-referral results than indiscriminate testing over all analyses. Conclusion: Targeted anti-CCP testing is a potentially cost-effective strategy to identify patients at risk of RA and enable prompt referral. Investigation of long-term costs and outcomes is warranted to determine the optimal testing strategy.

Bethany Shinkins and colleagues outline a consensus set of principles and evidence requirements for evaluating multicancer detection tests in screening programmes to inform UK policy

Objective: To study the performance of two contemporary sets of estimating equations for glomerular filtration rate, published by CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) and EKFC (European Kidney Function Consortium) that include one (creatinine or cystatin C only) and combined (creatinine and cystatin C) biomarkers, to assess their accuracy in a population with moderate chronic kidney disease. Design: Prospective cohort study. Setting: Primary, secondary, and tertiary care in six centres in England. Participants were recruited from April 2014 to January 2017. Participants: 1167 adults, aged ≥18 years, with moderate chronic kidney disease (estimated glomerular filtration rate 30-59 mL/min/1.73 m ² sustained over at least three months before recruitment). Main outcome measures: Accuracy of estimating equations CKD-EPI _creatinine , CKD-EPI _cystatin , CKD-EPI _{creatinine-cystatin} , EKFC _creatinine , EKFC _cystatin , and EKFC _{creatinine-cystatin} compared with measured glomerular filtration rate (iohexol clearance). Remodelled 2021 versions of the CKD-EPI equations were also studied. Accuracy was expressed as P30 (percentage of estimates within 30% of measured glomerular filtration rate). Results: Median age was 67.5 years, 58.3% of patients were men, 86.9% were white participants, and 27.8% had diabetes. Median measured glomerular filtration rate was 47.0 mL/min/1.73 m ² ; 57.0% of participants had albuminuria. Test calibration critically affected measurement of cystatin C. After recalibration of cystatin C, P30 values were 90.2% (CKD-EPI _creatinine ), 89.5% (CKD-EPI _cystatin ), 94.9% (CKD-EPI _{creatinine-cystatin} ), 88.0% (CKD-EPI(2021) _creatinine ), 94.9% (CKD-EPI(2021) _{creatinine-cystatin} ), 89.4% (EKFC _creatinine ), 91.0% (EKFC _cystatin ), and 94.9% (EKFC _{creatinine-cystatin} ). Creatinine based equations showed varying bias depending on the glomerular filtration rate level; inclusion of cystatin C in the equations improved this effect. Differences in accuracy in age, sex, and glomerular filtration rate level subgroups varied by equation. Equations combining creatinine and cystatin performed equally across age, sex, diabetes status, albuminuria status, and body mass index categories. Conclusions: The CKD-EPI _creatinine equation had acceptable accuracy in a white population in England with moderate chronic kidney disease. Combined dual biomarker equations showed higher accuracy than the CKD-EPI _creatinine equation and their equivalent creatinine only equations. Further research is needed to determine the most accurate equation to use in people of black and South Asian origin living in England.

Background: Newborn screening using whole genome sequencing is being evaluated in numerous projects across the world, including Genomics England Limited’s Generation Study. It presents considerable challenges for policy advisors, not least, given the logistics of simultaneously evaluating the evidence for the suggested 200 rare genetic conditions. The ‘genotype-first’ approach has the potential for harms through overdiagnosis, and benefits are uncertain. Objective: To assess different approaches to evaluating whole genome sequencing for newborn screening to inform the development of a robust method of evaluation for informing policy decisions. Methods: We approached the objective with systematic review methods for a sample of five conditions (considering gene penetrance, expressivity, accuracy and effectiveness of whole genome sequencing and effect of earlier treatment) (search inception to November 2023), evaluated the National Institutes of Health [US] Clinical Genome Resource (ClinGen) as an alternative evidence source for the five conditions and we compared this to a review of genomic studies of newborn screening cohorts reporting penetrance for pathogenic variants of any paediatric condition (search inception to February 2024). We undertook a methodological review of economic evaluations of whole genome sequencing/whole exome sequencing (search inception to January 2024) and explored public views on evaluating whole genome sequencing. Data sources: MEDLINE (Ovid), EMBASE (Ovid), Web of Science, Science Citation Index (via Clarivate), the Cochrane Library (via Wiley), cost-effectiveness analysis registry and American Economic Association electronic bibliography. Actionability reports and scores from the Clinical Genome Resource website (downloaded 30 April 2024). Results: The traditional review approach identified 268 studies reporting the genetic spectrum of individuals with the five conditions or benefits of earlier, symptomatic treatment. No evidence on the penetrance and expressivity or the accuracy or effectiveness of whole genome sequencing in newborns was identified. A review of 200 conditions would take a team of five reviewers 23 years to complete. Clinical Genome Resource reviews were available for four or five conditions. All four ‘actionability’ ratings disagreed with the findings of our reviews. Our review of 14 genomic studies of newborn screening cohorts found insufficient information to allow individual highly penetrant pathogenic variants for any condition to be identified. None of the 86 economic evaluations of whole genome sequencing or whole exome sequencing were set in a screening context. Some micro-costing studies are available that could help understand the resource use and costs associated with whole genome sequencing. Following a series of patient and public involvement meetings, attendees appreciated the uncertainties of whole genome sequencing. A wider stakeholder perspective is needed to inform policy decisions. Limitations: Although we only examined five conditions in depth, the consistency in lack of data suggests that our conclusions are robust. Conclusions: The systematic review approach for evaluating whole genome sequencing of newborns identified a paucity of high-quality evidence. Extending the review to all 200 conditions is not feasible. Currently, the use of existing genome resources and review of genomic studies of newborn screening cohorts are not viable alternatives. The cost-effectiveness of whole genome sequencing in a newborn screening context is unknown. Future work: Large-scale collaborative research is required to evaluate the short- and long-term harms, benefits and economic implications of whole genome sequencing for screening newborns. We propose a staged approach to evaluation, considering only conditions with pathogenic variants with high penetrance to minimise harm from overdiagnosis. Study registration: This study is registered as PROSPERO CRD42023475529. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis Programme (NIHR award ref: NIHR159928) and is published in full in Health Technology Assessment ; Vol. 29, No. 65. See the NIHR Funding and Awards website for further award information.

Objective: To evaluate systematic review approaches to synthesising evidence for policy advisers who are considering whether to screen newborns for hundreds of rare diseases using whole genome sequencing. Design: Series of systematic reviews and roadmap for evidence generation for policy advisers. Data sources: Medline, Embase, Science Citation Index, Cochrane Library. Methods: 200 conditions included in Genomics England's Generation Study were stratified into five groups and one condition randomly selected from each group using criteria designed to maximise variability and availability of data. 30 systematic reviews were undertaken (five conditions, six review questions) about penetrance, detection rate, accuracy, benefit of earlier treatment, and benefits and harms of screening for the five conditions (search from inception to November 2023). Results were synthesised and reviewer time recorded. Genomic studies of newborn screening cohorts that reported penetrance were systematically reviewed using a non-condition specific approach (search inception to January 2024). The conditions and genes selected for reporting by these studies were identified. ClinGen was explored for synthesising evidence. All approaches were assessed by considering review effort and level and quality of evidence. Results: The five conditions selected for systematic review were pyridoxine dependent epilepsy, heritable retinoblastoma, X linked hypophosphataemic rickets, familial haemophagocytic lymphohistiocytosis, and medium chain acyl-CoA dehydrogenase deficiency. 19 689 titles were screened and 268 papers included that addressed two of six research questions (detection rate and treatment benefit). No studies were identified for the remaining four research questions. Total reviewer time for five conditions was seven months. A team of five reviewers would take over 20 years to conduct similar reviews for 200 conditions. 10 published genomic studies of newborn screening cohorts were identified with a total of 76 268 newborns. The number of conditions screened for varied from 74 to 903, with low concordance (two of 1453 genes were included in all 10 studies). Selection of conditions was primarily based on clinical opinion. All studies reported and acted on genetic findings considered clinically significant, preventing collection of penetrance data. Conclusions: Current evidence synthesis methods are neither feasible nor fruitful to provide policy advisers like the UK National Screening Committee with the evidence needed to understand the benefits and harms of newborn screening for multiple conditions using whole genome sequencing because the evidence is not available to synthesise. Evidence should be created through studies that only report pathogenic variants to parents and clinicians where penetrance and expressivity have been established through empirical evidence rather than clinical opinion. A roadmap for future evidence generation is proposed to highlight the need for new evidence generation combined with a staged approach to evaluation, focusing on pathogenic variants with evidence of high penetrance and expressivity. Registration: Review registration PROSPERO (CRD42023475529).

Background: Early in the COVID-19 pandemic, there was concern about potentially unnecessary antibiotic prescribing in the National Health Service. Procalcitonin testing was being used in some hospitals to guide antibiotic use. This study aimed to investigate the impact of procalcitonin testing on United Kingdom’s antibiotic prescribing and health outcomes. Methods: Mixed-methods study comprising quantitative, qualitative and health economic work packages, including a: survey of National Health Service hospitals to understand procalcitonin use retrospective, controlled, interrupted time series analysis of aggregated, organisation-level data, including antibiotic dispensing, hospital activity and procalcitonin testing from acute hospital trusts/hospitals in England/Wales. Primary outcome: change in level and/or trend of antibiotic prescribing rates following introduction of procalcitonin multicentre, retrospective, cohort study of 5960 patients using patient-level clinical data from 11 trusts/health boards to determine the difference in early antibiotic prescribing between COVID-19 patients who did/did not have baseline procalcitonin testing by using propensity score matching. Primary outcome: days of early antibiotic therapy qualitative study exploring the decision-making process around antibiotic use for inpatients with COVID-19 pneumonia to identify the contextual factors, feasibility and acceptability of procalcitonin testing algorithms health economic analysis evaluating the cost-effectiveness of baseline procalcitonin testing using the matched data within a decision-analytic model. Setting: Acute hospital trusts/health boards in England/Wales. Participants: Inpatients ≥ 16 years, admitted to participating trusts/health boards and with a confirmed positive COVID-19 test between 1 February 2020 and 30 June 2020, National Health Service healthcare workers. Results: Early in the COVID-19 pandemic, procalcitonin use was expanded/introduced in many National Health Service hospitals, with variation in guidance and interpretation of results. The number of hospitals using procalcitonin in emergency/acute admissions rose from 17 (11%) to 74/146 (50.7%), and its use in intensive care unit increased from 70 (47.6%) to 124/147 (84.4%). Introduction of procalcitonin testing in emergency departments/acute medical admission units was associated with a statistically significant decrease in antibiotic use, which was not sustained. Patient-level data showed that baseline procalcitonin testing was associated with an average reduction in early antibiotic prescribing of 0.43 days (95% confidence interval: 0.22 to 0.64 days, p < 0.001) and a reduction of 0.72 days (95% confidence interval: 0.06 to 1.38 days, p = 0.03) in total antibiotic prescribing, with no increased mortality/hospital length of stay. Interviews revealed concerns about secondary bacterial infections that led to increased antibiotic prescribing in COVID-19 patients. As experience increased, clinician’s ability to distinguish between COVID-19 alone and bacterial coinfections increased. Antibiotic prescribing decisions were influenced by factors such as senior support, situational factors and organisational influences. The health economic analysis concluded that baseline procalcitonin testing was more likely to be cost-effective than not, albeit with some uncertainty. Conclusion: Baseline procalcitonin testing appears to have been an effective antimicrobial stewardship tool during the first wave of the pandemic, reducing antibiotic prescribing without evidence of harm. Limitations: The retrospective, hospital record-based studies were limited by missing data, incorrectly recorded information and lack of randomisation. Interviews with clinicians were conducted more than a year after the first wave, potentially resulting in recall bias. Future work: This study highlights the need for adaptive, inclusive, wide-reaching trials of infection diagnostics and implementation research to assess clinical utility before routine introduction into clinical practice. Funding: This synopsis presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number NIHR132254.

This methodological review identifies challenges in the development of health economic evaluations of newborn bloodspot screening (NBS) interventions and their consideration in NBS policy making. A systematic review of health economics methodological studies in NBS and stakeholder consultation was undertaken. The intervention under examination was defined as health economic decision analytic modelling used as decision support to NBS policy makers. An iterative search strategy was used to identify studies, and a data extraction framework was based upon a simple decision analytic model structure for the NBS decision problem. Synthesis was facilitated by two stakeholder workshops, which focused on ensuring the complete identification of challenges and developing recommendations. Sixteen methodological studies were identified. Data were extracted on challenges in decision criteria, decision variables, decision problem scope, defining model structure, selecting modelling method, the target condition, the screening test/protocol, outcome nodes, and other categories. Recommendations are made concerning supporting NBS decision making, NBS economic model structure and methods, data and estimation of model parameters, and overarching considerations. Recommendations for decision processes and methods research are put forward for the consideration of NBS policy makers and commissioners of research.

Figure S4 shows the Evaluation of the utility of regarding the 95% CI RR for late-stage incidence as a “forecast” of the final trial result for cancer-specific mortality, only including 15 trial arm comparisons with the late-stage outcome RR reported prior to the primary analysis for mortality but after the screening period had ended.

Supplementary Methods and Tables: includes five sections. Section 1 outlines the comparison with previously published meta-analyses of mortality outcome surrogacy in cancer screening trials (including Table S1 and Table S2, and description of this work). Section 2 outlines the systematic searches in bibliographic databases and study selection (including Tables S3-S11 and description of this work). Section 3 provides detailed information on trials and the reported outcomes for the assessment of mortality outcome surrogacy (including Table S12-15, and a reference list for those four tables). Section 4 details the detailed meta-analytic results, including sensitivity and subgroup analyses (Including the methods that surround Figures S1-4, Table S16, Table S17). Section 5 includes selected aspects of limitations in the reported trial data (Table S19 and a description of this work).