Alan Girling

Stepped‐wedge studies usually entail regular sampling of clusters over time. Yet the precision of the treatment effect estimator can sometimes be improved if the regular sampling scheme is replaced by one with preferential allocation of observations to particular time‐epochs within each cluster. We present some exact results for optimizing the allocation for a general experimental layout under a mixed effects model with a time‐varying cluster‐autocorrelation structure, together with an algorithm for generating optimal allocations. An index of cluster variation is introduced, an increasing function of both the intra‐class correlation and the total sample size, which encapsulates the influence of cluster‐level variation on the optimal allocation. For any specified layout there is a sampling scheme (the ‘best natural allocation’) that solves the optimization problem for all values of this index up to a threshold value which depends only on the cluster autocorrelations. Under such a scheme the treatment effect estimator is equal to a simple difference between the means of the treated and control observations. Best natural allocations stand alongside conventional parallel and cross‐over designs in giving equal weight to observations from all participants, even under stepped‐wedge layouts with irreversible interventions. When applied to a recent study of primary care training programmes in low‐ and middle‐ income countries (The REaCH study), the results lead to substantial reductions in total sample size, without loss of precision. For stepped‐wedge layouts with block‐exchangeable or time‐decaying cluster autocorrelations, we present explicit conditions for the optimality of staircase‐type sampling schemes, which can arise as best natural allocations in such cases.

Background: There is a need to identify approaches to reduce medication errors. Interest has converged on ePrescribing systems that incorporate computerised provider order entry and clinical decision support functionality. Objectives: We sought to describe the procurement, implementation and adoption of basic and advanced ePrescribing systems; to estimate their effectiveness and cost-effectiveness; and to develop a toolkit for system integration into hospitals incorporating implications for practice from our research. Design: We undertook a theoretically informed, mixed-methods, context-rich, naturalistic evaluation. Setting: We undertook six longitudinal case studies in four hospitals (sites C, E, J and K) that did not have ePrescribing systems at the start of the programme (three of which went live and one that never went live) and two hospitals (sites A and D) with embedded systems. In the three hospitals that implemented systems, we conducted interviews pre implementation, shortly after roll-out and at 1 year post implementation. In the hospitals that had embedded systems, we conducted two rounds of interviews, 18 months apart. We undertook a three-round eDelphi exercise involving 20 experts to identify 80 clinically important prescribing errors, which were developed into the Investigate Medication Prescribing Accuracy for Critical error Types (IMPACT) tool. We elicited the cost of an ePrescribing system at one (non-study) site and compared this with the calculated ‘headroom’ (the upper limit that the decision-maker should pay) for the systems (sites J, K and S) for which effectiveness estimates were available. We organised four national conferences and five expert round-table discussions to contextualise and disseminate our findings. Intervention: The implementation of ePrescribing systems with either computerised provider order entry or clinical decision support functionality. Main outcome measures: Error rates were calculated using the IMPACT tool, with changes over time represented as ratios of error rates (as a proportion of opportunities for errors) using Poisson regression analyses. Results: We conducted 242 interviews and 32.5 hours of observations and collected 55 documents across six case studies. Implementation was difficult, particularly in relation to integration and interfacing between systems. Much of the clinical decision support functionality in embedded sites remained switched off because of concerns about over alerting. Getting systems operational meant that little attention was devoted to system optimisation or secondary uses of data. The prescriptions of 1244 patients were audited pre computerised provider order entry and 1178 post computerised provider order entry implementation of system A at sites J and K, and system B at site S. A total of 21,138 opportunities for error were identified from 28,526 prescriptions. Across the three sites, for those prescriptions for which opportunities for error were identified, the error rate was found to reduce significantly post computerised provider order entry implementation, from 5.0% to 4.0% (p < 0.001). Post implementation, the overall proportion of errors (per opportunity) decreased significantly in sites J and S, but remained similar in site K, as follows: 4.3% to 2.8%, 7.4% to 4.4% and 4.0% to 4.4%, respectively. Clinical decision support implementation by error type was found to differ significantly between sites, ranging from 0% to 88% across clinical contraindication, dose/frequency, drug interactions and other error types (p < 0.001). Overall, 43 out of 78 (55%) of the errors had some degree of clinical decision support implemented in at least one of the hospitals. For the site in which no improvement was detected in prescribing errors (i.e. site K), the ePrescribing system represented a cost to the service for no countervailing benefit. Cost-effectiveness rose in proportion to reductions in error rates observed in the other sites (i.e. sites J and S). When a threshold value of £20,000 was used to define the opportunity cost, the system would need to cost less than £4.31 per patient per year, even in site S, where effectiveness was greatest. We produced an ePrescribing toolkit (now recommended for use by NHS England) that spans the ePrescribing life cycle from conception to system optimisation. Limitations: Implementation delays meant that we were unable to employ the planned stepped-wedge design and that the assessment of longer-term consequences of ePrescribing systems was impaired. We planned to identify the complexity of ePrescribing implementation in a number of contrasting environments, but the small number of sites means that we have to infer findings from this programme with considerable care. The lack of transparency regarding system costs is a limitation of our method. As with all health economic analyses, our analysis is subject to modelling assumptions. The research was undertaken in a modest number of early adopters, concentrated on high-risk prescribing errors and may not be generalisable to other hospitals. Conclusions: The implementation of ePrescribing systems was challenging. However, when fully implemented the ePrescribing systems were associated with a reduction in clinically important prescribing errors and our model suggests that such an effect is likely to be more cost-effective when clinical decision support is available. Careful system configuration considering clinical processes and workflows is important to achieving these potential benefits and, therefore, our findings may not be generalisable to all system implementations. Future work: Formative and summative evaluations of efforts will be central to promote learning across settings. Other priorities emerging from this work include the possibility of learning from international experiences and the commercial sector. Funding: This project was funded by the National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 10, No. 7. See the NIHR Journals Library website for further project information.

Background: NHS England’s 7-day services policy comprised 10 standards to improve access to quality health care across all days of the week. Six standards targeted hospital specialists on the assumption that their absence caused the higher mortality associated with weekend hospital admission: the ‘weekend effect’. The High-intensity Specialist-Led Acute Care (HiSLAC) collaboration investigated this using the implementation of 7-day services as a ‘natural experiment’. Objectives: The objectives were to determine whether or not increasing specialist intensity at weekends improves outcomes for patients undergoing emergency hospital admission, and to explore mechanisms and cost-effectiveness. Design: This was a two-phase mixed-methods observational study. Year 1 focused on developing the methodology. Years 2–5 included longitudinal research using quantitative and qualitative methods, and health economics. Methods: A Bayesian systematic literature review from 2000 to 2017 quantified the weekend effect. Specialist intensity measured over 5 years used self-reported annual point prevalence surveys of all specialists in English acute hospital trusts, expressed as the weekend-to-weekday ratio of specialist hours per 10 emergency admissions. Hospital Episode Statistics from 2007 to 2018 provided trends in weekend-to-weekday mortality ratios. Mechanisms for the weekend effect were explored qualitatively through focus groups and on-site observations by qualitative researchers, and a two-epoch case record review across 20 trusts. Case-mix differences were examined in a single trust. Health economics modelling estimated costs and outcomes associated with increased specialist provision. Results: Of 141 acute trusts, 115 submitted data to the survey, and 20 contributed 4000 case records for review and participated in qualitative research (involving interviews, and observations using elements of an ethnographic approach). Emergency department attendances and admissions have increased every year, outstripping the increase in specialist numbers; numbers of beds and lengths of stay have decreased. The reduction in mortality has plateaued; the proportion of patients dying after discharge from hospital has increased. Specialist hours increased between 2012/13 and 2017/18. Weekend specialist intensity is half that of weekdays, but there is no relationship with admission mortality. Patients admitted on weekends are sicker (they have more comorbid disease and more of them require palliative care); adjustment for severity of acute illness annuls the weekend effect. In-hospital care processes are slightly more efficient at weekends; care quality (errors, adverse events, global quality) is as good at weekends as on weekdays and has improved with time. Qualitative researcher assessments of hospital weekend quality concurred with case record reviewers at trust level. General practitioner referrals at weekends are one-third of those during weekdays and have declined further with time. Limitations: Observational research, variable survey response rates and subjective assessments of care quality were compensated for by using a difference-in-difference analysis over time. Conclusions: Hospital care is improving. The weekend effect is associated with factors in the community that precede hospital admission. Post-discharge mortality is increasing. Policy-makers should focus their efforts on improving acute and emergency care on a ‘whole-system’ 7-day approach that integrates social, community and secondary health care. Future work: Future work should evaluate the role of doctors in hospital and community emergency care and investigate pathways to emergency admission and quality of care following hospital discharge. Funding: This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 13. See the NIHR Journals Library website for further project information.

In this article, we review and evaluate a number of methods used in the design and analysis of small three‐arm parallel cluster randomized trials. We conduct a simulation‐based study to evaluate restricted randomization methods including covariate‐constrained randomization and a novel method for matched‐group cluster randomization. We also evaluate the appropriate modelling of the data and small sample inferential methods for a variety of treatment effects relevant to three‐arm trials. Our results indicate that small‐sample corrections are required for high (0.05) but not low (0.001) values of the intraclass correlation coefficient and their performance can depend on trial design, number of clusters, and the nature of the hypothesis being tested. The Satterthwaite correction generally performed best at an ICC of 0.05 with a nominal type I error rate for single‐period trials, and in trials with repeated measures type I error rates were between 0.04 and 0.06. Restricted randomization methods produce little benefit in trials with repeated measures but in trials with single post‐intervention design can provide relatively large gains in power when compared to the most unbalanced possible allocations. Matched‐group randomization improves power but is not as effective as covariate‐constrained randomization. For model‐based analysis, adjusting for fewer covariates than were used in a restricted randomization process under any design can produce non‐nominal type I error rates and reductions in power. Where comparisons to two‐arm cluster trials are possible, the performance of the methods is qualitatively very similar.

Background: Emergency abdominal surgery is associated with poor patient outcomes. We studied the effectiveness of a national quality improvement (QI) programme to implement a care pathway to improve survival for these patients.Objectives: The objectives were to assess whether or not the QI programme improves 90-day survival after emergency abdominal surgery; to assess effects on 180-day survival, hospital stay and hospital readmission; and to better understand these findings through an integrated process evaluation, ethnographic study and cost-effectiveness analysis.Design: This was a stepped-wedge cluster randomised trial. Hospitals were organised into 15 geographical clusters, and commenced the QI programme in random order over 85 weeks. Analyses were performed on an intention-to-treat basis. The primary outcome was analysed using a mixed-effects parametric survival model, adjusting for time-related effects. Ethnographic and economics data were collected in six hospitals. The process evaluation included all hospitals.Setting: The trial was set in acute surgical services of 93 NHS hospitals.Participants: Patients aged ≥ 40 years who were undergoing emergency abdominal surgery were eligible.Intervention: The intervention was a QI programme to implement an evidence-based care pathway.Main outcome measures: The primary outcome measure was mortality within 90 days of surgery. Secondary outcomes were mortality within 180 days, length of hospital stay and hospital readmission within 180 days. The main economic measure was the quality-adjusted life-years.Data sources: Data were obtained from the National Emergency Laparotomy Audit database; qualitative interviews and ethnographic observations; quality-of-life and NHS resource use data were collected via questionnaires.Results: Of 15,873 eligible patients from 93 NHS hospitals, primary outcome data were analysed for 8482 participants in the usual care group and 7374 in the QI group. The primary outcome occurred in 1393 participants in the usual care group (16%), compared with 1210 patients in the QI group (16%) [QI vs. usual care hazard ratio (HR) 1.11, 95% confidence interval (CI) 0.96 to 1.28]. No differences were found in mortality at 180 days or hospital readmission; there was a small increase in hospital stay in the QI group (HR for discharge 0.90, 95% CI 0.83 to 0.97). There were only modest improvements in care processes following QI implementation. The ethnographic study revealed good QI engagement, but limited time and resources to implement change, affecting which processes teams addressed, the rate of change and eventual success. In some sites, there were challenges around prioritising the intervention in busy environments and in obtaining senior engagement. The intervention is unlikely to be cost-effective at standard cost-effectiveness thresholds, but may be cost-effective over the lifetime horizon.Limitations: Substantial delays were encountered in securing data access to national registries. Fewer patients than expected underwent surgery and the mortality rate was lower than anticipated.Conclusions: There was no survival benefit from a QI programme to implement a care pathway for patients undergoing emergency abdominal surgery. The modest impact of the intervention on process measures, despite good clinician engagement, may have been limited by the time and resources needed to improve patient care.Future work: Future QI programmes must balance intervention complexity with the practical realities of NHS services to ensure that such programmes can be delivered with the resources available.Trial registration: Current Controlled Trials ISRCTN80682973 andThe Lancetprotocol 13PRT/7655.Funding: This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full inHealth Services and Delivery Research; Vol. 7, No. 32. See the NIHR Journals Library website for further project information.

Cluster randomised trials with unequal sized clusters often have lower precision than with clusters of equal size. To allow for this, sample sizes are inflated by a modified version of the design effect for clustering. These inflation factors are valid under the assumption that randomisation is stratified by cluster size. We investigate the impact of unequal cluster size when that constraint is relaxed, with particular focus on the stepped-wedge cluster randomised trial, where this is more difficult to achieve. Assuming a multi-level mixed effect model with exchangeable correlation structure for a cross-sectional design, we use simulation methods to compare the precision for a trial with clusters of unequal size to a trial with clusters of equal size (relative efficiency). For a range of scenarios we illustrate the impact of various design features (the cluster-mean correlation – a function of the intracluster correlation and the cluster size, the number of clusters, number of randomisation sequences) on the average and distribution of the relative efficiency. Simulations confirm that the average reduction in precision, due to varying cluster sizes, is smaller in a stepped-wedge trial compared to the parallel trial. However, the variance of the distribution of the relative efficiency is large; and is larger under the stepped-wedge design compared to the parallel design. This can result in large variations in actual power, depending on the allocation of clusters to sequences. Designs with larger variations in cluster sizes, smaller number of clusters and studies with smaller cluster-mean correlations (smaller cluster sizes or smaller intra-cluster correlation) are particularly at risk. The actual realised power in a stepped-wedge trial might be substantially higher or lower than that estimated. This is particularly important when there are a small number of clusters or the variability in cluster sizes is large. Constraining the randomisation on cluster size, where feasible, might mitigate this effect.

Objective To determine whether the higher weekend admission mortality risk is attributable to increased severity of illness. Design Retrospective analysis of 4 years weekend and weekday adult emergency admissions to a university teaching hospital in England. Outcome measures 30-day postadmission weekend:weekday mortality ratios adjusted for severity of illness (baseline National Early Warning Score (NEWS)), routes of admission to hospital, transfer to the intensive care unit (ICU) and demographics. Results Despite similar emergency department daily attendance rates, fewer patients were admitted on weekends (mean admission rate 91/day vs 120/day) because of fewer general practitioner referrals. Weekend admissions were sicker than weekday (mean NEWS 1.8 vs 1.7, p=0.008), more likely to undergo transfer to ICU within 24 hours (4.2% vs 3.0%), spent longer in hospital (median 3 days vs 2 days) and less likely to experience same-day discharge (17.2% vs 21.9%) (all p values Conclusions Patients admitted to hospital on weekends are sicker than those admitted on weekdays. The cause of the weekend effect may lie in community services.

Background: A cluster trial with unequal cluster sizes often has lower precision than one with equal clusters, with a corresponding inflation of the design effect. For parallel group trials, adjustments to the design effect are available under sampling models with a single intracluster correlation. Design effects for equal clusters under more complex scenarios have appeared recently (including stepped wedge trials under cross‐sectional or longitudinal sampling). We investigate the impact of unequal cluster size in these more general settings.Results: Assuming a linear mixed model with an exchangeable correlation structure that incorporates cluster and subject autocorrelation, we compute the relative efficiency (RE) of a trial with clusters of unequal size under a size‐stratified randomization scheme, as compared to an equal cluster trial with the same total number of observations. If there are no within‐cluster time effects, the RE exceeds that for a parallel trial. In general, the RE is a weighted average of the RE for a parallel trial and the RE for a crossover trial in the same clusters. Existing approximations for parallel designs are extended to the general setting. Increasing the cluster size by the factor (1 + CV²), where CV is the coefficient of variation of cluster size, leads to conservative sample sizes, as in a popular method for parallel trials.Conclusion: Methods to assess experimental precision for single‐period parallel trials with unequal cluster sizes can be extended to stepped wedge and other complete layouts under longitudinal or cross‐sectional sampling. In practice, the loss of precision due to unequal cluster sizes is unlikely to exceed 12%.

Quality of cardiopulmonary resuscitation (CPR) is associated with survival, but recommended guidelines are often not met, and less than half the children with an in-hospital arrest will survive to discharge. A single-center before-and-after study demonstrated that outcomes may be improved with a novel training program in which all pediatric intensive care unit staff are encouraged to participate in frequent CPR refresher training and regular, structured resuscitation debriefings focused on patient-centric physiology. This ongoing trial will assess whether a program of structured debriefings and point-of-care bedside practice that emphasizes physiologic resuscitation targets improves the rate of survival to hospital discharge with favorable neurologic outcome in children receiving CPR in the intensive care unit. This study is designed as a hybrid stepped-wedge trial in which two of ten participating hospitals are randomly assigned to enroll in the intervention group and two are assigned to enroll in the control group for the duration of the trial. The remaining six hospitals enroll initially in the control group but will transition to enrolling in the intervention group at randomly assigned staggered times during the enrollment period. To our knowledge, this is the first implementation of a hybrid stepped-wedge design. It was chosen over a traditional stepped-wedge design because the resulting improvement in statistical power reduces the required enrollment by 9 months (14%). However, this design comes with additional challenges, including logistics of implementing an intervention prior to the start of enrollment. Nevertheless, if results from the single-center pilot are confirmed in this trial, it will have a profound effect on CPR training and quality improvement initiatives. ClinicalTrials.gov, NCT02837497 . Registered on July 19, 2016.

Background In this UK study, we investigated the impact of computerised physician order entry (CPOE) and clinical decision support (CDS) implementation on the rate of 78 high-risk prescribing errors amenable to CDS. Methods We conducted a preintervention/postintervention study in three acute hospitals in England. A predefined list of prescribing errors was incorporated into an audit tool. At each site, approximately 4000 prescriptions were reviewed both pre-CPOE and 6 months post-CPOE implementation. The number of opportunities for error and the number of errors that occurred were collated. Error rates were then calculated and compared between periods, as well as by the level of CDS. Results The prescriptions of 1244 patients were audited pre-CPOE and 1178 post-CPOE implementation. A total of 28 526 prescriptions were reviewed, with 21 138 opportunities for error identified based on 78 defined errors. Across the three sites, for those prescriptions where opportunities for error were identified, the error rate was found to reduce significantly post-CPOE implementation, from 5.0% to 4.0% (PConclusions Implementation of CPOE with CDS was associated with clinically important reductions in the rate of high-risk prescribing errors. Given the pre-post design, these findings however need to be interpreted with caution. The occurrence of errors was found to be highly dependent on the level of restriction of CDS presented to the prescriber, with the effect that different configurations of the same CPOE system can produce very different results.