Low stakes, high risks: the problem of intertemporal validity of PISA in Latin America
Authors:
- 4 December 2019
- journal article
- research article
- Published by Taylor & Francis in Journal of Education Policy
- Vol. 36 (2) , 279-302
Abstract
Since 2000, the PISA test implemented by OECD has become the prime benchmark for international comparisons in education. The 2015 PISA edition introduced methodological changes that altered the nature of its results. PISA made no longer valid non-reached items of the final part of the test, assuming that those unanswered questions were more a problem of motivation than skills. Latin American countries were the only ones that had a large proportion of non-reached items in the first PISA editions. The methodological change had a large impact in these countries and was miscommunicated by the OECD reports. This makes the first two PISA editions (2000 and 2003) no longer comparable with the latest ones (2006, 2009, 2012 and 2015). We analyze hypotheses for the Latin American exception in PISA, suggesting that the introduction of the new Computer-Based Assessment of PISA in 2015, combined with an increasing culture of evaluation and political influences damaged intertemporal comparisons of PISA. Although PISA is a low-stakes assessment for students, it has become high-stakes for governments. This triggered mechanisms to increase test-taking motivation, and students tended to answer more questions of the test, but at the same time, these changes made intertemporal comparisons flawed.Keywords
Funding Information
- Instituto Natura (Project “Map of Educational Policy in Latin Americ)
This publication has 52 references indexed in Scilit:
- The Socio‐Economic Gradient in Teenagers' Reading Skills: How Does England Compare with Other Countries?*Fiscal Studies, 2012
- Profiles of Students’ Interest in Science Issues around the World: Analysis of data from PISA 2006International Journal of Science Education, 2011
- Skill and will: test‐taking motivation and assessment qualityAssessment in Education: Principles, Policy & Practice, 2010
- Can Differential Rapid-Guessing Behavior Lead to Differential Item Functioning?International Journal of Testing, 2010
- Governing by numbers: the PISA ‘effect’ in EuropeJournal of Education Policy, 2009
- Are There Test Administrator Effects in Large-Scale Educational Assessments?Methodology, 2007
- Low Examinee Effort in Low-Stakes Assessment: Problems and Potential SolutionsEducational Assessment, 2005
- Educational Assessments in Latin America: The State of the ArtApplied Psychology, 2004
- Test theory and language-learning assessmentLanguage Testing, 1995
- Motivational determinants of risk-taking behavior.Psychological Review, 1957