Input as a Key Element in Test Design: A Narrative of Designing an Innovative Critical Thinking Assessment
Open Access
- 3 May 2024
- journal article
- Published by King Mongkut's University of Technology Thonburi in rEFLections
- Vol. 31 (2) , 516-542
Abstract
Test input has often been taken as a given in test design practice. Nearly all guides for test designers provide extensive coverage of how to design test items but pay little attention to test input. This paper presents the case that test input plays a crucial role in designing tests of soft skills that have rarely been assessed in existing tests. In the process of designing a test of critical thinking, several attempts following existing test design guides resulted in poor tests that did not truly assess the intended objectives. These initial attempts used the norm of short passages as test input. Following these failures, we switched to using real-world input, such as tweets, numerical tables, and spam emails. In doing so, it was found that a particular input type favored a particular sub-skill of critical thinking and a particular item type. For example, using tweets as input enabled the assessment of the Perspective Taking sub-skill of critical thinking. This paper concludes that in designing skill tests, integrating appropriate input is at least as important as item design and calls for reevaluating the functions of test input as a distinct and dynamic element.Keywords
This publication has 36 references indexed in Scilit:
- Creativity and Critical Thinking2021
- Critical Thinking Skills2017
- Assessing critical thinking in higher education: the HEIghten™ approach and preliminary validity evidenceAssessment & Evaluation in Higher Education, 2016
- A rich interpretation of numeracy for the 21st century: a survey of the state of the fieldZDM – Mathematics Education, 2015
- Assessing Critical Thinking in Higher Education: Current State and Directions for Next‐Generation AssessmentETS Research Report Series, 2014
- Teacher beliefs about feedback within an assessment for learning environment: Endorsement of improved learning over student well-beingTeaching and Teacher Education, 2012
- Halpern Critical Thinking Assessment Predicts Real‐World Outcomes of Critical ThinkingApplied Cognitive Psychology, 2012
- Rethinking college readinessNew Directions for Higher Education, 2008
- Threats to the Validity of Locally Developed Multiple-Choice Tests in Medical Education: Construct-Irrelevant Variance and Construct UnderrepresentationAdvances in Health Sciences Education, 2002
- Teaching critical thinking for transfer across domains: Disposition, skills, structure training, and metacognitive monitoring.American Psychologist, 1998