Trusting teachers’ judgement: research evidence of the reliability and validity of teachers’ assessment used for summative purposes
Author:
- 1 September 2005
- journal article
- research article
- Published by Taylor & Francis in Research Papers in Education
- Vol. 20 (3) , 245-270
Abstract
This paper summarizes the findings of a systematic review of research on the reliability and validity of teachers’ assessment used for summative purposes. In addition to the main question, the review also addressed the question ‘What conditions affect the reliability and validity of teachers’ summative assessment?’ The initial search for studies meeting the explicit inclusion criteria of relevance found 431potentially relevant studies. This number was gradually reduced, through the systematic review procedures, to 30 studies, which specifically addressed the review questions. These studies were subject to in‐depth data extraction conducted independently by two researchers, followed by reconciliation of any differences of interpretation. This procedure was also used to judge the weight of evidence provided for the review by each study so that greater emphasis could be given to findings from the most relevant and methodologically sound research. The findings of the review by no means constitute a ringing endorsement of teachers’ assessment; there was evidence of low reliability and bias in teachers’ judgements made in certain circumstances. However, this has to be considered against the low validity and lower than generally assumed reliability of external tests. The findings also point to ways of overcoming the deficiencies of teachers’ assessment and lead to implications for assessment policy, practice and research, which are proposed in the final section of the paper.Keywords
This publication has 36 references indexed in Scilit:
- Public Examinations in Hong KongAssessment in Education: Principles, Policy & Practice, 1999
- Large‐scale Portfolio Assessments in the US: evidence pertaining to the quality of measurementAssessment in Education: Principles, Policy & Practice, 1998
- Comparing Teacher Assessment and Standard Task Results in England: the relationship between pupil characteristics and attainmentAssessment in Education: Principles, Policy & Practice, 1998
- The Construct Validity and Context Dependency of Teacher Assessment of Practical Skills in Some Pre‐university Level Science ExaminationsAssessment in Education: Principles, Policy & Practice, 1996
- Assessing, Recording and Reporting Students’ Educational Progress: the case for ‘subject profiles’Assessment in Education: Principles, Policy & Practice, 1996
- Models of teacher assessment among primary school teachers in EnglandThe Curriculum Journal, 1996
- Teachers’ assessments of primary children's classroom work in the creative artsEducational Research, 1996
- Standards and Science Education in Scottish SchoolsStudies in Science Education, 1995
- Teachers' Estimates of Candidates' Performances in Public ExaminationsAssessment in Education: Principles, Policy & Practice, 1995
- An investigation into the accuracy of A‐level predicted gradesEducational Research, 1994