MHMatt Homer

Matt Homer

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Matt Homer has a total of 77 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Choosing School Lunches in Primary School – Insights From Children, Catering Professionals and Observation

    by H. Abbotts‐Holmes, A. Turnbull, K. Stronge, M. Homer, H. Ensaff

    Introduction: A better understanding of children's food choice behaviour, including within school settings, is needed to inform effective approaches to address recognised shortfalls in children's dietary intakes. The aim of this study was to explore pupils' food choice in school and decision‐making processes underpinning the selection of school lunches. Methods: Five Primary schools in North West England were recruited. Direct observation of lunchtimes and in‐class selection of meals was conducted, followed by semi‐structured focus group discussions with 119 children (7–11 years) and interviews with four catering managers. Reflexive thematic analysis was used to provide themes and subthemes. Results: Familiarity and repeated exposure, alongside taste preferences were important drivers for selecting a school lunch – with children, in most cases, the primary decision‐maker. Pupils spoke about the lunch costs, its relevance to household finances and deciding between a school or packed lunch. Children articulated that individual preferences were more important than choosing the same as peers. In some cases, nevertheless, conversations with peers led children to try new foods. Conclusions: Findings highlight modifiable behavioural determinants, including familiarity – and point to further work to support children in exploring and expanding their repertoire of ‘acceptable’ meals. These include opportunities to leverage meal descriptions and images, to promote a greater diversity of meals chosen. Children and parents, as key stakeholders, are essential to informing and shaping such changes.

  • Open Access

    What drives postgraduate awarding gaps? Modelling master’s degree outcomes

    by Matt Homer, Huiming Ding, Layla Kouara

    Educational disparities in the UK’s Higher Education (HE) system have long been a source of concern (Broecke & Nicholls, 2006; Connor et al., 1996; Naylor & Smith, 2004; Owen, 2000; Richardson, 200...

  • Open Access

    Evidencing improvement in examiner calibration in OSCEs

    by Matt Homer, Vlad Ababei

    In developing and administering OSCE-type assessments, institutions can spend significant resources training examiners and designing stations/scoring instruments in attempts to ensure that they are well-calibrated. This paper, situated in the context of a high-stakes OSCE for international medical graduates wanting to work in the national health service in the UK, employs a recently developed quantitative measure of examiner calibration to identify which stations show relatively high and low average degrees of calibration between examiners. Using documentary analysis of station materials, we then investigate these stations qualitatively to better understand what factors might drive better calibration as stations develop and design elements change over time. We find that those stations that are better calibrated are typically newer with more detailed and relevant scoring guidance and support materials, whilst there is little evidence that the nature of the task(s) or other contextual factors are important in determining calibration levels. In this work, we provide strong evidence of how key developments in station design, the quality of support materials and enhanced examiner training practices can succeed in improving degrees of calibration in OSCE stations - and suggest ways that all institutions might improve their practices in this regard.


  • Open Access

    Going beyond hawks and doves – Measuring degrees of examiner misalignment in OSCEs

    by Matt Homer

    Minimising examiner differences in scoring in OSCEs is key in supporting the validity of the assessment outcomes. This is particularly true for common OSCE designs where the same station is administered across parallel circuits, with examiners nested within these. However, the common classification of extreme examiners as 'hawks' or 'doves' can be overly simplistic. Rather, it is the difference in patterns of scoring across circuits that better indicates poor levels of agreement between examiners that can unfairly advantage particular groups of candidates in comparison with others in different circuits. In this paper, a new measure of differences in examiner scoring is presented that quantifies the different combined patterns of scoring in global grades and checklist/domain scores for pairs of examiners assessing in the same station but in different circuits. Based on calculating the area between separate examiners' individual borderline regression lines, this measure can be used as a post hoc metric to provide a broad range of validity evidence for the assessment and its outcomes. In challenging the 'hawks'/'doves' paradigm, this work presents a detailed empirical analysis of a new misalignment metric in a particular high-stakes context and gives a range of evidence of its contribution to overall OSCE quality control processes and of improved fairness to candidates over time. The paper concludes with comments on developing the metric to contexts where there are multiple parallel circuits which will allows its practical application to a broader set of OSCE contexts.


  • Open Access

    Tailoring support following summative assessments: a latent profile analysis of student outcomes across five medical specialities

    by Huiming Ding, Matt Homer

    Summative assessments are often underused for feedback, despite them being rich with data of students’ applied knowledge and clinical and professional skills. To better inform teaching and student support, this study aims to gain insights from summative assessments through profiling students’ performance patterns and identify those students missing the basic knowledge and skills in medical specialities essential for their future career. We use Latent Profile Analysis to classify a senior undergraduate year group (n = 295) based on their performance in applied knowledge test (AKT) and OSCE, in which items and stations are pre-classified across five specialities (e.g. Acute and Critical Care, Paediatrics,…). Four distinct groups of students with increasing average performance levels in the AKT, and three such groups in the OSCE are identified. Overall, these two classifications are positively correlated. However, some students do well in one assessment format but not in the other. Importantly, in both the AKT and the OSCE there is a mixed group containing students who have met the required standard to pass, and those who have not. This suggests that a conception of a borderline group at the exam-level can be overly simplistic. There is little literature relating AKT and OSCE performance in this way, and the paper discusses how our analysis gives placement tutors key insights into providing tailored support for distinct student groups needing remediation. It also gives additional information to assessment writers about the performance and difficulty of their assessment items/stations, and to wider faculty about student overall performance and across specialities.


  • Open Access

    Towards a more nuanced conceptualisation of differential examiner stringency in OSCEs

    by Matt Homer

    Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In this work, a large candidate-level exam dataset is analysed to develop a more sophisticated understanding of examiner stringency. Station scores are modelled based on global grades—with each candidate, station and examiner allowed to vary in their ability/stringency/difficulty in the modelling. In addition, examiners are also allowed to vary in how they discriminate across grades—to our knowledge, this is the first time this has been investigated. Results show that examiners contribute strongly to variance in scoring in two distinct ways—via the traditional conception of score stringency (34% of score variance), but also in how they discriminate in scoring across grades (7%). As one might expect, candidate and station account only for a small amount of score variance at the station-level once candidate grades are accounted for (3% and 2% respectively) with the remainder being residual (54%). Investigation of impacts on station-level candidate pass/fail decisions suggest that examiner differential stringency effects combine to give false positive (candidates passing in error) and false negative (failing in error) rates in stations of around 5% each but at the exam-level this reduces to 0.4% and 3.3% respectively. This work adds to our understanding of examiner behaviour by demonstrating that examiners can vary in qualitatively different ways in their judgments. For institutions, it emphasises the key message that it is important to sample widely from the examiner pool via sufficient stations to ensure OSCE-level decisions are sufficiently defensible. It also suggests that examiner training should include discussion of global grading, and the combined effect of scoring and grading on candidate outcomes.


  • Open Access

    Setting defensible minimum-stations-passed standards in OSCE-type assessments

    by Matt Homer

    Alongside the usual exam-level cut-score requirement, the use of a conjunctive minimum number of stations passed (MNSP) standard in OSCE-type assessments is common practice across some parts of the world. Typically, the MNSP is fixed in advance with little justification, and does not vary from one administration to another in a particular setting—which is not congruent to best assessment practice for high stakes examinations. In this paper, we investigate empirically four methods of setting such a standard in an examinee-centred (i.e. post hoc) and criterion-based way that allows the standard to vary appropriately with station and test difficulty. Using many administrations (n = 442) from a single exam (PLAB2 in the UK), we show via mixed modelling that the total number of stations passed for each candidate has reliability close to that of the total test score (relative g-coefficient 0.73 and 0.76 respectively). We then argue that calculating the MNSP based on the predicted number of stations passed at the ‘main’ exam-level cut-score (i.e. for the borderline candidate) is conceptually, theoretically and practically preferred amongst the four approaches considered. Further analysis indicates that this standard does vary from administration to administration, but acts in a secondary way, with approximately a quarter of exam-level candidate failures resulting from application of the MNSP standard alone. Collectively, this work suggests that employing the identified approach to setting the MNSP standard is practically possible and, in many settings, is more defensible than using a fixed number of stations set in advance.


  • Open Access

    Pass/fail decisions and standards: the impact of differential examiner stringency on OSCE outcomes

    by Matt Homer

    Variation in examiner stringency is a recognised problem in many standardised summative assessments of performance such as the OSCE. The stated strength of the OSCE is that such error might largely balance out over the exam as a whole. This study uses linear mixed models to estimate the impact of different factors (examiner, station, candidate and exam) on station-level total domain score and, separately, on a single global grade. The exam data is from 442 separate administrations of an 18 station OSCE for international medical graduates who want to work in the National Health Service in the UK. We find that variation due to examiner is approximately twice as large for domain scores as it is for grades (16% vs. 8%), with smaller residual variance in the former (67% vs. 76%). Combined estimates of exam-level (relative) reliability across all data are 0.75 and 0.69 for domains scores and grades respectively. The correlation between two separate estimates of stringency for individual examiners (one for grades and one for domain scores) is relatively high (r=0.76) implying that examiners are generally quite consistent in their stringency between these two assessments of performance. Cluster analysis indicates that examiners fall into two broad groups characterised as hawks or doves on both measures. At the exam level, correcting for examiner stringency produces systematically lower cut-scores under borderline regression standard setting than using the raw marks. In turn, such a correction would produce higher pass rates—although meaningful direct comparisons are challenging to make. As in other studies, this work shows that OSCEs and other standardised performance assessments are subject to substantial variation in examiner stringency, and require sufficient domain sampling to ensure quality of pass/fail decision-making is at least adequate. More, perhaps qualitative, work is needed to understand better how examiners might score similarly (or differently) between the awarding of station-level domain scores and global grades. The issue of the potential systematic bias of borderline regression evidenced for the first time here, with sources of error producing cut-scores higher than they should be, also needs more investigation.


  • Open Access

    “I was told it would help with my Psychology”: Do post-16 Core Maths qualifications in England support other subjects?

    by Rachel Mathieson, Matt Homer

    Continuing to study mathematics throughout schooling is considered important in most developed countries, where mathematics is incorporated within the curriculum until school-leaving age. By comparison, in England, relatively few post-16 (upper secondary) students study mathematics once it becomes optional. Core Maths qualifications, introduced in England in 2014, are intended to help increase post-16 mathematics participation. This paper uses data from a three-year, mixed-methods, longitudinal study to investigate a perceived benefit of the qualifications: the support Core Maths might give to other curriculum subjects in post-16 students’ programmes. Amongst teachers and students, we find a widespread conviction that studying Core Maths benefits students’ other subjects contemporaneously, whilst in national data from Core Maths students we find no evidence yet of enhanced examination attainment in other qualifications. We suggest that Core Maths impacts positively on students in ways which could be more usefully, and accurately, emphasised in promoting the course.