Evidence-Based Design Approach for Assessing Universal Competencies in Higher Education: Advantages and Features
Open Access
- 31 January 2025
- journal article
- Published by Moscow Polytechnic University in Vysshee Obrazovanie v Rossii = Higher Education in Russia
- Vol. 34 (1) , 82-105
Abstract
The article addresses the issue of assessing universal competencies, which are becoming increasingly important given modern demands on university graduates. It emphasizes the need for valid and reliable tools to measure skills such as critical thinking, creativity, communication, and complex problem-solving. The paper presents an analysis of an evidence-based design approach, which integrates cognitive theories, innovative educational technologies, and psychometric models to achieve objective assessments of these competencies.Particular attention is given to the evidence-centered design methodology, which relies on scenario-based tasks closely reflecting real-life and professional situations. This method emphasizes constructing valid argument that link students’ observable behaviors to claims about their competence levels. The approach not only captures demonstrations of knowledge and skills in realistic contexts but also structures the instrument development process to ensure that empirical results are theoretically sound. A distinctive feature of this approach is the process of building arguments from collected evidence, differentiating it from traditional methods, which focus primarily on the statistical and psychometric properties of tests.The principles of this method are illustrated using the CT Test for assessing university students’ critical thinking, a tool with established psychometric quality, which has been piloted with over 10,000 students in Russian universities. The article argues that implementing evidence-based tools will substantially enhance the validity and reliability of competency assessments among students, making this approach a promising candidate for integration into educational practice.Keywords
This publication has 33 references indexed in Scilit:
- Sociocognitive and Argumentation Perspectives on Psychometric Modeling in Educational AssessmentPsychometrika, 2024
- Exploring social and cognitive dimensions of collaborative problem solving in an open online simulation-based taskComputers in Human Behavior, 2020
- International Performance Assessment of Learning in Higher Education (iPAL): Research and Development2018
- Principled Approaches to Assessment Design, Development, and Implementation2016
- A methodology for applying students' interactive task performance scores from a multimedia-based performance assessment in a Bayesian NetworkComputers in Human Behavior, 2016
- Assessing Critical Thinking in Higher Education: Current State and Directions for Next‐Generation AssessmentETS Research Report Series, 2014
- The identification and validation process of proportional reasoning attributes: an application of a cognitive diagnosis modeling frameworkMathematics Education Research Journal, 2013
- The curriculum-instruction-assessment triadEmpirical Research in Vocational Education and Training, 2012
- 31A Review of Cognitively Diagnostic Assessment and a Summary of Psychometric Models2006
- Evidence and Inference in Educational AssessmentPsychometrika, 1994