Self-, peer-, and teacher-assessments in Japanese university EFL writing classrooms
Author:
- 1 January 2009
- journal article
- Published by SAGE Publications in Language Testing
- Vol. 26 (1) , 075-100
Abstract
Multifaceted Rasch measurement was used in the present study with 91 student and 4 teacher raters to investigate how self- and peer-assessments work in comparison with teacher assessments in actual university writing classes. The results indicated that many self-raters assessed their own writing lower than predicted. This was particularly true for high-achieving students. Peer-raters were the most lenient raters; however, they rated high-achieving writers lower and low-achieving writers higher. This tendency was independent of their own writing abilities and therefore offered no support for the hypothesis that high-achieving writers rated severely and low-achieving writers rated leniently. On the other hand, most peer-raters were internally consistent and produced fewer bias interactions than self- and teacher-raters. Each of the four teachers was internally consistent; however, each displayed a unique bias pattern. Self-, peer-, and teacher-raters assessed Grammar severely and Spelling leniently. The analysis also revealed that teacher-raters assessed Spelling, Format, and Punctuation differently from the other criteria. It was concluded that self-assessment was somewhat idiosyncratic and therefore of limited utility as a part of formal assessment. Peer-assessors on the other hand were shown to be internally consistent and their rating patterns were not dependent on their own writing performance. They also produced relatively few bias interactions. These results suggest that in at least some contexts, peer-assessments can play a useful role in writing classes. By using multifaceted Rasch measurement, teachers can inform peer-raters of their bias patterns and help them develop better quality assessment criteria, two steps that might lead to better quality peer-assessment.Keywords
This publication has 33 references indexed in Scilit:
- Peer assessment of language proficiencyLanguage Testing, 2005
- A many-facet Rasch analysis of the second language group oral discussion taskLanguage Testing, 2003
- Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research ProcessAssessment & Evaluation in Higher Education, 2002
- Having second thoughts: Student perceptions before and after a peer assessment exerciseStudies in Higher Education, 1997
- Peer Assessment by Groups of Group WorkAssessment & Evaluation in Higher Education, 1995
- Further Developments in Peer Assessment of Group ProjectsAssessment & Evaluation in Higher Education, 1994
- Comparing Teacher and Student Responses to Written WorkTESOL Quarterly, 1994
- Peer Assessment of an Individual ‘s Contribution to a Group ProjectAssessment & Evaluation in Higher Education, 1993
- DEVELOPMENT OF A PEER ASSESSMENT TECHNIQUE FOR OBTAINING INDIVIDUAL MARKS ON A GROUP PROJECTAssessment & Evaluation in Higher Education, 1990
- The construct validation of self-ratings of communicative language abilityLanguage Testing, 1989