In the rating scales of major international language tests, as well as in automated evaluation systems (e.g. e-rater), a positive relationship is often claimed between lexical diversity, holistic quality of written or spoken discourses, and language proficiency of candidates. This article reports a posteriori validation study that analysed a sample of the archived data of an international language test to examine empirically to what extent such relationships exist. It is also noted that previous studies on lexical diversity in the field of applied linguistics have focused exclusively on either written or spoken discourses, no study to date has compared lexical diversity of spoken and written discourses produced by the same participants. Therefore, the second aim of this article is to understand the differences in lexical diversity between writing and speaking task performances, and to what extent the topics of the writing prompts may affect lexical diversity of written discourses. Using D as a measure of lexical diversity (Malvern and Richards 1997, 2002; Malvern et al. 2004), it was found that D had a statistically significant and positive correlation with the overall quality ratings of both writing and speaking performances as well as the candidates’ general language proficiency. Nevertheless, the significant relationships were not borne out across the subgroups of the sample in terms of gender, first language background, purpose of taking the test and topics of the writing prompts. The different writing topics also had significant effects on lexical diversity—especially the topics that candidates were highly familiar with—even after controlling for writing ability and overall language proficiency. The lexical diversity of candidates’ writing and speaking performances were approximately at the same level; further, D was found to be a better predictor of speaking than writing performance. The implications of these findings are discussed with specific reference to the use of lexical diversity measures to inform language test validation and the development of lexical diversity parameters in automated evaluation systems.

Publications with shared reference:

0 publications found