Stephen Senn

A simple device for balancing for a continuous covariate in clinical trials is to stratify by whether the covariate is above or below some target value, typically the predicted median. The object is to improve balance of the covariate and hence efficiency of the treatment estimate particularly if the trial is small, as may be the case if the disease in question is rare. This raises an issue as to which model should be used for modelling the effect of treatment on the outcome variable, Y . Should one fit, the stratum indicator, S , the continuous covariate, X , both or neither? When a covariate is added to a linear model there are three consequences for inference: (a) the mean square error effect, (b) the variance inflation factor and (c) second order precision. We consider that it is valuable to consider these three factors separately, even if, ultimately, it is their joint effect that matters. We present some simple theory, concentrating in particular on the variance inflation factor, that may be used to guide trialists in their choice of model. We also consider the case where the precise form of the relationship between the outcome and the covariate is not known. We conclude by recommending that the continuous covariate should always be in the model but that, depending on circumstances, there may be some justification in fitting the stratum indicator also.

We have studied 21 435 unique randomized controlled trials (RCTs) from the Cochrane Database of Systematic Reviews (CDSR). Of these trials, 7224 (34%) have a continuous (numerical) outcome and 14 211 (66%) have a binary outcome. We find that trials with a binary outcome have larger sample sizes on average, but also larger standard errors and fewer statistically significant results. We conclude that researchers tend to increase the sample size to compensate for the low information content of binary outcomes, but not sufficiently. In many cases, the binary outcome is the result of dichotomization of a continuous outcome, which is sometimes referred to as “responder analysis”. In those cases, the loss of information is avoidable. Burdening more participants than necessary is wasteful, costly, and unethical. We provide a method to convert a sample size calculation for the comparison of two proportions into one for the comparison of the means of the underlying continuous outcomes. This demonstrates how much the sample size may be reduced if the outcome were not dichotomized. We also provide a method to calculate the loss of information after a dichotomization. We apply this method to all the trials from the CDSR with a binary outcome, and estimate that on average, only about 60% of the information is retained after dichotomization. We provide R code and a shiny app at: https://vanzwet.shinyapps.io/info_loss/ to do these calculations. We hope that quantifying the loss of information will discourage researchers from dichotomizing continuous outcomes. Instead, we recommend they “model continuously but interpret dichotomously”. For example, they might present “percentage achieving clinically meaningful improvement” derived from a continuous analysis rather than by dichotomizing raw data.

A causal model has been proposed for including external controls in randomized clinical trials that is based on five critical assumptions and has been illustrated using the example of two trials in spinal muscular atrophy. It is argued that four of these assumptions are obviously false in the example used and that this would be a problem for many cases where one sought to use the approach. The claimed superiority to Bayesian methods is debatable, since trial effects have not been allowed for. It is concluded that trialists should be cautious in applying such an approach to using external controls.

N-of-1 trials are defined and the popular paired cycle design is introduced, together with an explanation as to how suitable sequences may be constructed.Various approaches to analysing such trials are explained and illustrated using a simulated data set. It is explained how choosing an appropriate analysis depends on the question one wishes to answer. It is also shown that for a given question, various equivalent approaches to analysis can be found, a fact which may be exploited to expand the possible software routines that may be used.Sets of N-of-1 trials are analogous to sets of parallel group trials. This means that software for carrying out meta-analysis can be used to combine results from N-of-1 trials. In doing so, it is necessary to make one important change, however. Because degrees of freedom for estimating variances for individual subjects will be scarce, it is advisable to estimate local standard errors using pooled variances. How this may be done is explained and fixed and random effect approaches to combining results are illustrated.