Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods
- 5 October 2020
- journal article
- research article
- Published by Taylor & Francis in Assessment in Education: Principles, Policy & Practice
- Vol. 27 (6) , 635-654
Abstract
Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we compared two scoring methods for metacognitive knowledge in reading based on pair-wise comparisons of strategies and with conventional Likert-scale responses of selected items. Metacognitive knowledge scored with conventional Likert-scale responses demonstrated higher cross-cultural comparability than the pair-wise comparison method. Linked with reading competence, motivation and control strategy in reading, scores from the two scoring methods showed differential criterion validity, possibly related to the types of tasks (understanding and remembering versus summarising), item content (complexity and discrimination between preferred strategies in reading) and common method variance (e.g., individuals’ stable response style in rating scales). Theoretical and methodological implications are discussed..Keywords
This publication has 38 references indexed in Scilit:
- A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go.Psychological Bulletin, 2011
- Diagnose von Strategiewissen beim TextverstehenZeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 2009
- Cultural biases: a non‐Anglophone perspectiveAssessment in Education: Principles, Policy & Practice, 2008
- Reading strategy use and motivation among Chinese good and poor readers in Hong KongJournal of Research in Reading, 2003
- Evaluating Goodness-of-Fit Indexes for Testing Measurement InvarianceStructural Equation Modeling: A Multidisciplinary Journal, 2002
- Methodological Issues in Psychological Research on CultureJournal of Cross-Cultural Psychology, 2000
- Approaches to Learning across Cultures: The role of assessmentAssessment in Education: Principles, Policy & Practice, 1999
- A Closer Look at Learning Strategies, L2 Proficiency, and GenderTESOL Quarterly, 1995
- Construct Validation of MetacognitionThe Journal of Psychology, 1993
- Children's Metacognition About Reading: issues in Definition, Measurement, and InstructionEducational Psychologist, 1987