Effects of training on raters of ESL compositions
Author:
- 1 July 1994
- journal article
- research article
- Published by SAGE Publications in Language Testing
- Vol. 11 (2) , 197-223
Abstract
Several effects of training on composition raters have been hypothesized but not investigated empirically. This article presents an analysis of the verbal protocols of four inexperienced raters of ESL placement compositions scoring the same essays both before and after rater training. The verbal protocols show that training clarified the intended scoring criteria for raters, modified their expectations of student writing and provided a reference group of other raters with which raters could compare themselves, although agreement with peers was not an over-riding concern. These results are generally in accordance with hypothesized effects of rater training.Keywords
This publication has 10 references indexed in Scilit:
- Effects of training on raters of ESL compositionsLanguage Testing, 1994
- Protocol Analysis1993
- Expertise in evaluating second language compositionsLanguage Testing, 1990
- Reliability, Validity, and Holistic Scoring: What We Know and What We Need to KnowCollege Composition and Communication, 1990
- Researching Practice: Evaluating Assessment EssaysCollege Composition and Communication, 1986
- Holistic Evaluation of ESL Compositions: Can It Be Validated Objectively?TESOL Quarterly, 1984
- The Validity of Using Holistic Scoring to Evaluate Writing: A Critical OverviewResearch in the Teaching of English, 1984
- Written Composition: Toward a Theory of EvaluationCollege English, 1980
- Telling more than we can know: Verbal reports on mental processes.Psychological Review, 1977
- An Investigation of the Reliability of Five Procedures for Grading English ThemesResearch in the Teaching of English, 1967