Harvey Goldstein

Education systems around the world increasingly rely on school value-added models to monitor school performance and hold schools to account. These models typically focus on a limited number of academic outcomes. We explore how the traditional multilevel modelling approach to school value-added models can be extended to simultaneously analyse multiple academic and non-academic outcomes and the implications of doing so for systems using student data to monitor schools and inform school accountability. We jointly model student attainment, absence, and exclusion data for schools in England. We find very different results across the three outcomes, in terms of the size and consistency of school effects, and the importance of adjusting for student and school characteristics. The results suggest the three outcomes are capturing fundamentally distinct aspects of school performance, all of which are therefore important for education systems to monitor and explore including in systems of school accountability.

A first step when fitting multilevel models to continuous responses is to explore the degree of clustering in the data. Researchers fit variance-component models and then report the proportion of variation in the response that is due to systematic differences between clusters. Equally they report the response correlation between units within a cluster. These statistics are popularly referred to as variance partition coefficients (VPCs) and intraclass correlation coefficients (ICCs). When fitting multilevel models to categorical (binary, ordinal, or nominal) and count responses, these statistics prove more challenging to calculate. For categorical response models, researchers appeal to their latent response formulations and report VPCs/ICCs in terms of latent continuous responses envisaged to underly the observed categorical responses. For standard count response models, however, there are no corresponding latent response formulations. More generally, there is a paucity of guidance on how to partition the variation. As a result, applied researchers are likely to avoid or inadequately report and discuss the substantive importance of clustering and cluster effects in their studies. A recent article drew attention to a little-known exact algebraic expression for the VPC/ICC for the special case of the two-level random-intercept Poisson model. In this article, we make a substantial new contribution. First, we derive exact VPC/ICC expressions for more flexible negative binomial models that allows for overdispersion, a phenomenon which often occurs in practice. Then we derive exact VPC/ICC expressions for three-level and random-coefficient extensions to these models. We illustrate our work with an application to student absenteeism. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Within-individual variability of repeatedly measured exposures might predict later outcomes (e.g., blood pressure (BP) variability (BPV) is an independent cardiovascular risk factor above and beyond mean BP). Because 2-stage methods, known to introduce bias, are typically used to investigate such associations, we introduce a joint modeling approach, examining associations of mean BP and BPV across childhood with left ventricular mass (indexed to height; LVMI) in early adulthood with data (collected 1990–2011) from the UK Avon Longitudinal Study of Parents and Children cohort. Using multilevel models, we allowed BPV to vary between individuals (a “random effect”) as well as to depend on covariates (allowing for heteroskedasticity). We further distinguished within-clinic variability (“measurement error”) from visit-to-visit BPV. BPV was predicted to be greater at older ages, at higher body weights, and in female participants and was positively correlated with mean BP. BPV had a weak positive association with LVMI (10% increase in within-individual BP variance was predicted to increase LVMI by 0.21%, 95% credible interval: −0.23, 0.69), but this association became negative (−0.78%, 95% credible interval: −2.54, 0.22) once the effect of mean BP on LVMI was adjusted for. This joint modeling approach offers a flexible method of relating repeatedly measured exposures to later outcomes.

The presence of randomly distributed measurement errors in scale scores such as those used in educational and behavioural assessments implies that careful adjustments are required to statistical model estimation procedures if inferences are required for ‘true’ as opposed to ‘observed’ relationships. In many cases this requires the use of external values for ‘reliability’ statistics or ‘measurement error variances’ which may be provided by a test constructor or else inferred or estimated by the data analyst. Popular measures are those described as ‘internal consistency’ estimates and sometimes other measures based on data grouping. All such measures, however, make particular assumptions that may be questionable but are often not examined. In this paper we focus on scaled scores derived from aggregating a set of indicators, and set out a general methodological framework for exploring different ways of estimating reliability statistics and measurement error variances, critiquing certain approaches and suggesting more satisfactory methods in the presence of longitudinal data. In particular, we explore the assumption of local (conditional) item response independence and show how a failure of this assumption can lead to biased estimates in statistical models using scaled scores as explanatory variables. We illustrate our methods using a large longitudinal data set of mathematics test scores from Queensland, Australia. Key messages The paper explores different approaches to obtaining good estimates of reliability for scale scores. The assumptions behind existing methods for reliability estimation are reviewed and critiqued. The paper proposes instrumental variable techniques whose assumptions can be satisfied using longitudinal data. The suggested procedures are applied to a large longitudinal educational data set from Australia.

The requirement to anonymise data sets that are to be released for secondary analysis should be balanced by the need to allow their analysis to provide efficient and consistent parameter estimates. The proposal in this article is to integrate the process of anonymisation and data analysis. The first stage uses the addition of random noise with known distributional properties to some or all variables in a released (already pseudonymised) data set, in which the values of some identifying and sensitive variables for data subjects of interest are also available to an external ‘attacker’ who wishes to identify those data subjects in order to interrogate their records in the data set. The second stage of the analysis consists of specifying the model of interest so that parameter estimation accounts for the added noise. Where the characteristics of the noise are made available to the analyst by the data provider, we propose a new method that allows a valid analysis. This is formally a measurement error model and we describe a Bayesian MCMC algorithm that recovers consistent estimates of the true model parameters. A new method for handling categorical data is presented. The article shows how an appropriate noise distribution can be determined.

This paper proposes an alternative educational accountability system for England that moves away from simplistic comparisons, or league tables, among schools towards a more nuanced reporting at a level of education authorities or other school groupings. Based upon the sampling of pupils within schools, it proposes the use of quantitative pupil assessment data as screening devices that provide evidence within an integrated accountability system that includes inspection. At the same time, it aims to provide a richer set of data than currently available for research as well as accountability purposes. We argue that if carefully implemented within a context of school improvement, such a system has the potential to largely eliminate the deleterious side effects and curriculum distortions of the present system. While being proposed within the context of the current English secondary school system, the proposals will have relevance for other phases of schooling and similar systems in other countries.

Within-individual variability of repeatedly-measured exposures may predict later outcomes: e.g. blood pressure (BP) variability (BPV) is an independent cardiovascular risk factor above and beyond mean BP. Since two-stage methods, known to introduce bias, are typically used to investigate such associations, we introduce a joint modelling approach, examining associations of both mean BP and BPV across childhood to left ventricular mass (indexed to height; LVMI) in early adulthood with data from the UK’s Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. Using multilevel models, we allow BPV to vary between individuals (a “random effect”) as well as to depend on covariates (allowing for heteroscedasticity). We further distinguish within-clinic variability (“measurement error”) from visit-to-visit BPV. BPV was predicted to be greater at older ages, at higher bodyweights, and in females, and was positively correlated with mean BP. BPV had a positive association with LVMI (10% increase in SD(BP) was predicted to increase LVMI by mean = 0.42% (95% credible interval: −0.47%, 1.38%)), but this association became negative (mean = −1.56%, 95% credible interval: −5.01%, 0.44%)) once the effect of mean BP on LVMI was adjusted for. This joint modelling approach offers a flexible method of relating repeatedly-measured exposures to later outcomes.

(2019). PISA and the globalisation of education: a critical commentary on papers published in AIE special issue 4/2019. Assessment in Education: Principles, Policy & Practice. Ahead of Print.

Pseudonymisation is one element of a range of measures that can be used to protect the privacy of individuals. ‘Pseudonymisation at source’ is a technique used by data providers to avoid identification of individuals before data are linked for secondary uses such as service evaluation or research. The technique involves replacement of direct identifiers, known as ‘personal data’ or ‘confidential patient information’, such as NHS number, date of birth and postcode, with a pseudonym, which does not reveal a person’s real world identity. Use of the same pseudonymisation key for multiple data sources before data are shared enables data sources to be linked together without using ‘personal data’ and therefore avoids the need for patient consent or other legal provision under the Data Protection Act or the General Data Protection Regulation.¹ As we discuss below, however, this limits the utility and quality of any resulting linked datasets.