BNBen Naismith

Ben Naismith

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Ben Naismith has a total of 23 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    A Systematic Review of Test Component Ordering with Implications for Language Assessment

    by Ben Naismith, Ramsey L. Cardwell

    How to optimize ordering the parts of a language test (e.g., items, tasks, and sections) is often overlooked or assumed to be self-evident. However, ordering choices are an essential component of test design as they may impact test takers’ objective performance, perceived performance, or affective states. In this paper, we report on a systematic review of test component ordering research, focusing on 88 studies from 1933 to 2023. We provide a narrative synthesis, describing typical outcome variables (e.g., test-taker performance), independent variables (e.g., order of difficulty), and mediating variables (e.g., language proficiency). Key findings, mostly from higher educational contexts, indicate that easy-to-hard ordering may lead to better performance, though these effects are mediated by various test-taker characteristics, especially anxiety and proficiency. While section ordering by language skills is common practice in language testing, there is scant empirical support for this approach, or for organizing tests by content or format. We discuss the implications of these findings for language test developers and suggest avenues for future research, particularly the need for more studies on ordering effects in language assessment contexts.


  • Open Access

    Beyond GPA and language proficiency: A systematic literature review of international students' academic success factors

    by Maria Kostromitina, Ben Naismith, Jill Burstein, Luke Plonsky

    The rapid increase in international students, many of whom study in a second language (L2), has driven extensive research that examines the contribution of L2 proficiency to students' academic success. However, focusing solely on L2 proficiency as captured by language test scores provides a limited perspective, ignoring the many factors that contribute to academic achievement. This systematic literature review aims to broaden our understanding of this topic by exploring additional factors that contribute to the academic success of international students. Analysing 54 peer‐reviewed studies from 1990 to 2023, the review highlights that while outcome measures of success remain largely focused on grade point average (GPA), researchers are increasingly investigating predictors other than language proficiency, such as (1) background characteristics (e.g. gender), (2) individual differences (e.g. personality), (3) previous educational experiences (e.g. high school grades), (4) current educational experiences (e.g. familiarity with the learning environment), and (5) current social experiences (e.g. sense of belonging), which display varying effect sizes on international student success. Findings from this study have implications for academic stakeholders, suggesting a more holistic approach to understanding and supporting the academic success of international students. Context and implications: Rationale for this study: This systematic review addresses a critical gap in understanding international student success. While existing research has focused heavily on language proficiency and GPA, this study offers a more holistic view of success factors, recognising that the international student population faces complex challenges beyond language barriers, and that student success is measured not only via GPA. Why the new findings matter: The findings challenge the narrow focus on GPA and language proficiency as prevalent metrics of international student success by revealing a broader range of factors that shape international student success. In addition, the study explores the quality of scholarship in the domain. Implications for practitioners, policy makers, and researchers: For researchers, this study highlights the need for more transparent research designs and reporting practices when investigating predictors of international student success. It is also recommended that research should expand into understudied areas in this domain, particularly looking at international education in non‐English speaking destinations and at the graduate level. For faculty, findings emphasise the importance of implementing culturally responsive teaching practices and creating inclusive learning environments. Finally, for university admissions, the study indicates that relying solely on language proficiency scores provides an incomplete picture of potential success, suggesting a more holistic admissions approach.


  • Open Access

    Developing an Automatic Pronunciation Scorer: Aligning Speech Evaluation Models and Applied Linguistics Constructs

    by Danwei Cai, Ben Naismith, Maria Kostromitina, Zhongwei Teng, Kevin P. Yancey, Geoffrey T. LaFlair

    Globalization and increases in the numbers of English language learners have led to a growing demand for English proficiency assessments of spoken language. In this paper, we describe the development of an automatic pronunciation scorer built on state‐of‐the‐art deep neural network models. The model is trained on a bespoke human‐rated dataset that reflects current perspectives on pronunciation and intelligibility. The new scorer is evaluated along three criteria: How well it explains expert human ratings, how it compares to other state‐of‐the‐art automatic pronunciation scorers in explaining expert human ratings, and the extent to which it exhibits bias toward different groups of test takers. Results indicate that the proposed scorer shows strong positive correlations with expert human ratings and outperforms other scorers. However, the scorer shows some bias related to audio quality and language family groups. We conclude with future directions for mitigating bias and argue that this scorer holds potential for use in operational settings.


  • Open Access

    The impact of collocational proficiency features on expert ratings of L2 English learners’ writing

    by Ben Naismith, Alan Juffs

    Lexical proficiency is a multifaceted phenomenon that greatly impacts human judgments of writing quality. However, the importance of collocations’ contribution to proficiency assessment has received less attention than that of single words, despite collocations’ essential role in language production. This study, therefore, investigated how aspects of collocational proficiency affect the ratings that examiners give to English learner essays. To do so, collocational features related to sophistication and accuracy were manipulated in a set of argumentative essays. Examiners then rated the texts and provided rationales for their choices. The findings revealed that the use of lower-frequency words significantly and positively impacted the experts’ ratings. When used as part of collocations, such words then provided a small yet significant additional boost to ratings. Notably, there was no significant effect for increased collocational accuracy. These findings suggest that low-frequency words within collocations are particularly salient to examiners and deserving of pedagogic focus.


  • Open Access

    Identifying and analyzing ‘noisy’ spelling errors in a second language corpus

    by Alan Juffs, Ben Naismith

    Alan Juffs, Ben Naismith. Proceedings of the Tenth Workshop on Noisy and User-generated Text. 2025.


  • “I'm not a walking dictionary”

    by Ben Naismith, Leo Selivan

    Language teachers’ self-efficacy (LTSE) beliefs encompass the many ways in which language teachers view their own capabilities to support language learning. Importantly, these means are task-, domain-, and context-specific, interacting with language teachers’ beliefs about their own language proficiency and their own language awareness. To date, there has been extensive research into LTSE beliefs and teacher cognition more generally, but specific language domains such as vocabulary have not received due attention. To help address this gap, this chapter reports on a mixed-methods investigation of LTSE beliefs about teaching vocabulary and the relationship of these beliefs to lexical awareness. Quantitative data were collected through an online survey and were supplemented with qualitative data from semi-structured interviews. The findings reveal that confidence in presenting/practising vocabulary and what we termed “responsively using teacher lexical knowledge” were the greatest predictors of overall vocabulary teaching confidence. Other trends that emerged related to beliefs about pre-teaching vocabulary, beliefs about using learners’ first language, and teachers’ awareness of multi-word lexical items. These findings have implications for teacher preparation, suggesting aspects of vocabulary instruction which may be currently underserved, uncovering characteristics of confident vocabulary teachers that can be emulated, and highlighting issues in vocabulary instruction which remain controversial.


  • Ensuring Fairness of Human- and AI-Generated Test Items

    by William C. M. Belzak, Ben Naismith, Jill Burstein


  • Assessing Speaking on the Duolingo English Test

    by Yena Park, Ramsey Cardwell, Sarah Goodwin, Ben Naismith, Geoffrey LaFlair, Kai-Ling Lo, Kevin Yancey