Matthew Snelling
Matthew Snelling has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Andrew Dawes
Universiteit van Kaapstad
2 shared publications Elizabeth Girdwood
Innovation Edge, Cape Town, South Africa
2 shared publicationsLinda Biersteker
Innovation Edge, Cape Town, South Africa
2 shared publicationsAndrew Dawes
Department of Psychology, University of Cape Town, South Africa
1 shared publicationC. G. Tredoux
1 shared publicationColin Tredoux
Department of Psychology, University of Cape Town, South Africa
1 shared publication
Latest Publications
- Open Access
- 3 October 2026
- Published by Center for Open Science in PsyArXiv
Measurement Invariance as an Item-Screening Criterion for Large Language Model Benchmarks: A Position Note
Item response theory is increasingly applied to large language model benchmarks. Recent work uses item response models to select small informative subsets of items, to adapt item administration to model ability, to flag likely label errors, and to estimate latent ability instead of relying on raw accuracy. Recent work also establishes that leaderboard orderings move when the item set changes, whether the items are reordered, replaced by an ability estimate, or removed after an audit. Each of the established uses screens items on a criterion estimated from responses to a fixed presentation of each item. This note argues for a fourth criterion, estimated across presentations instead of within one: an item is a candidate for removal when its response function is not invariant across a change to the test that carries no construct-relevant content. The note maps the configural, metric and scalar invariance hierarchy onto the parameters of a two-parameter logistic model, separates the test-level hierarchy from the item-level removal rule, specifies the dependence structure implied by testing the same models under several presentations, states four hypotheses, and states what would falsify the argument. To the author's knowledge, no prior study has tested a configural, metric and scalar item-parameter hierarchy across controlled presentations of the same large language models and used item-level failure as an exclusion criterion.
- 3 October 2026
- Published by Center for Open Science in PsyArXiv
Open AccessMeasurement Invariance as an Item-Screening Criterion for Large Language Model Benchmarks: A Position Note
Item response theory is increasingly applied to large language model benchmarks. Recent work uses item response models to select small informative subsets of items, to adapt item administration to model ability, to flag likely label errors, and to estimate latent ability instead of relying on raw accuracy. Recent work also establishes that leaderboard orderings move when the item set changes, whether the items are reordered, replaced by an ability estimate, or removed after an audit. Each of the established uses screens items on a criterion estimated from responses to a fixed presentation of each item. This note argues for a fourth criterion, estimated across presentations instead of within one: an item is a candidate for removal when its response function is not invariant across a change to the test that carries no construct-relevant content. The note maps the configural, metric and scalar invariance hierarchy onto the parameters of a two-parameter logistic model, separates the test-level hierarchy from the item-level removal rule, specifies the dependence structure implied by testing the same models under several presentations, states four hypotheses, and states what would falsify the argument. To the author's knowledge, no prior study has tested a configural, metric and scalar item-parameter hierarchy across controlled presentations of the same large language models and used item-level failure as an exclusion criterion.
- 29 November 2021
- Published by Taylor & Francis in Early Education and Development
- Vol. 34 (1) , 256-273
To What Extent Can Community-based Playgroup Programmes Targeting Low-income Children Improve Learning Outcomes Prior to Entering the Reception Year in South Africa? A Quasi-experimental Field Study
Research Findings: Community-based playgroups in low-and-middle-income countries increase access to early learning at lower cost than centers. However, evidence of effectiveness is limited. The effects of one, two, or three playgroup sessions per week on children’s performance on the Early Learning Outcome Measure (ELOM) (n = 112; baseline M = 55.32 months) were investigated using a quasi-experimental design. Data on children’s home learning environments was obtained from primary caregivers, and program staff provided information on playgroup characteristics. Multilevel modeling demonstrated significant differences between programs in ELOM score improvement over eight months participation (F = 6.48; p = .01); Fisher LSD Post Hoc pairwise comparisons indicated that three session playgroups produced significantly greater gains than two and one session models (p < .01). ELOM Total scores at baseline, and the quality of children’s interactions with adults and peers, predicted growth in ELOM Total scores by endline. Baseline age, height-for-age, child-practitioner ratio, and caregiver time for activities with their children, predicted growth in certain ELOM domains at endline. Practice or Policy: Until provision of quality center-based provision is realized, our findings indicate that well-designed community-based playgroups of sufficient dose, have the potential to improve learning outcomes and school readiness in children from poor households.
- 29 November 2021
- Published by Taylor & Francis in Early Education and Development
- Vol. 34 (1) , 256-273
To What Extent Can Community-based Playgroup Programmes Targeting Low-income Children Improve Learning Outcomes Prior to Entering the Reception Year in South Africa? A Quasi-experimental Field Study
Research Findings: Community-based playgroups in low-and-middle-income countries increase access to early learning at lower cost than centers. However, evidence of effectiveness is limited. The effects of one, two, or three playgroup sessions per week on children’s performance on the Early Learning Outcome Measure (ELOM) (n = 112; baseline M = 55.32 months) were investigated using a quasi-experimental design. Data on children’s home learning environments was obtained from primary caregivers, and program staff provided information on playgroup characteristics. Multilevel modeling demonstrated significant differences between programs in ELOM score improvement over eight months participation (F = 6.48; p = .01); Fisher LSD Post Hoc pairwise comparisons indicated that three session playgroups produced significantly greater gains than two and one session models (p < .01). ELOM Total scores at baseline, and the quality of children’s interactions with adults and peers, predicted growth in ELOM Total scores by endline. Baseline age, height-for-age, child-practitioner ratio, and caregiver time for activities with their children, predicted growth in certain ELOM domains at endline. Practice or Policy: Until provision of quality center-based provision is realized, our findings indicate that well-designed community-based playgroups of sufficient dose, have the potential to improve learning outcomes and school readiness in children from poor households.
- 25 January 2019
- Published by Wiley in Child: Care, Health and Development
- Vol. 45 (2) , 257-270
The development of a South African Early Learning Outcomes Measure: A South African instrument for measuring early learning program outcomes
Background: Assessment of early childhood development programme effectiveness in South Africa is hampered by a lack of suitable measures that account for variations in cultural and socio‐economic backgrounds and can be administered by non‐professionals. This contribution reports the standardisation of the South African Early Learning Outcomes Measure (ELOM), an instrument designed for population level monitoring of the developmental status of children aged 50–69 months and for evaluation of early learning programmes.Methods: The development of the ELOM was informed by South Africa's National Curriculum Framework from Birth to Four and its National Early Learning and Development Standards. ELOM items were drawn from reliable and valid instruments, particularly those used in Africa and other developing regions and were clustered in five domains: gross motor development, fine motor coordination and visual motor integration, emergent numeracy and mathematics, cognition and executive functioning, emergent literacy and language. The ELOM was standardised on a sample of 1,331 children aged 50–69 months, from five South African official languages and five socio‐economic strata. Item Response Theory techniques were used to establish reliability, validity, and differential item functioning.Results: Confirmatory Factor Analysis established that ELOM domains are unidimensional and internally consistent. Items discriminate reliably between more and less able children and do not discriminate unfairly between children of the same ability from different language backgrounds. Socio‐economic gradients were evident in children's performance. South African Early Learning Development Standards (ELDS) based on standard scores were developed and set at the 60th percentile of the sample standard score distribution.Conclusions: This research produced the first South African, age‐validated population‐level standardised instrument that can be administered relatively cheaply by trained non‐professionals. This will facilitate the assessment of the efficacy of early learning programmes in enabling children to reach ELDS prior to entering Grade R and track progress toward Sustainable Development Goal 4.2.
- 25 January 2019
- Published by Wiley in Child: Care, Health and Development
- Vol. 45 (2) , 257-270
The development of a South African Early Learning Outcomes Measure: A South African instrument for measuring early learning program outcomes
Background: Assessment of early childhood development programme effectiveness in South Africa is hampered by a lack of suitable measures that account for variations in cultural and socio‐economic backgrounds and can be administered by non‐professionals. This contribution reports the standardisation of the South African Early Learning Outcomes Measure (ELOM), an instrument designed for population level monitoring of the developmental status of children aged 50–69 months and for evaluation of early learning programmes.Methods: The development of the ELOM was informed by South Africa's National Curriculum Framework from Birth to Four and its National Early Learning and Development Standards. ELOM items were drawn from reliable and valid instruments, particularly those used in Africa and other developing regions and were clustered in five domains: gross motor development, fine motor coordination and visual motor integration, emergent numeracy and mathematics, cognition and executive functioning, emergent literacy and language. The ELOM was standardised on a sample of 1,331 children aged 50–69 months, from five South African official languages and five socio‐economic strata. Item Response Theory techniques were used to establish reliability, validity, and differential item functioning.Results: Confirmatory Factor Analysis established that ELOM domains are unidimensional and internally consistent. Items discriminate reliably between more and less able children and do not discriminate unfairly between children of the same ability from different language backgrounds. Socio‐economic gradients were evident in children's performance. South African Early Learning Development Standards (ELDS) based on standard scores were developed and set at the 60th percentile of the sample standard score distribution.Conclusions: This research produced the first South African, age‐validated population‐level standardised instrument that can be administered relatively cheaply by trained non‐professionals. This will facilitate the assessment of the efficacy of early learning programmes in enabling children to reach ELDS prior to entering Grade R and track progress toward Sustainable Development Goal 4.2.
- 2 January 2018
- Published by Taylor & Francis in Childhood Education
- Vol. 94 (1) , 12-16
Early Learning Assessment Innovation in South Africa: A locally appropriate monitoring tool
Innovation in education goes beyond curriculum and content delivery. We also must look for ways to ensure that how we assess our education programs is relevant and current.
- 2 January 2018
- Published by Taylor & Francis in Childhood Education
- Vol. 94 (1) , 12-16
Early Learning Assessment Innovation in South Africa: A locally appropriate monitoring tool
Innovation in education goes beyond curriculum and content delivery. We also must look for ways to ensure that how we assess our education programs is relevant and current.