Investigating construct validity of cognitive load measurement using single-item subjective rating scales
Open Access
- 3 December 2024
- journal article
- research article
- Published by Springer Nature in Instructional Science
- Vol. 53 (1) , 71-97
Abstract
Cognitive load studies are mostly centered on information on perceived cognitive load. Single-item subjective rating scales are the dominant measurement practice to investigate overall cognitive load. Usually, either invested mental effort or perceived task difficulty is used as an overall cognitive load measure. However, the extent to which the results of these two single-items differ has not yet been sufficiently investigated. Although subjective rating scales are widely used, they are criticized and questioned as their validity is doubted. This study examines construct validity of both cognitive load rating scales (invested mental effort, perceived task difficulty) using relative task difficulty and task demands (cognitive processes and availability of possible answer options) as criteria, adds further evidence supporting the validity of single-item subjective ratings as an indicator for overall cognitive load, and shows how ratings of cognitive load differ when the invested mental effort or the perceived task difficulty item is used. The results indicate that self-ratings might be influenced by the availability of possible answer options as well as cognitive processes necessary to work on a task. The findings also confirm the idea that self-ratings for perceived task difficulty and invested mental effort do not measure the same but different aspects of overall cognitive load. Furthermore, our findings clearly advise to precisely examine at which point and how frequently cognitive load is measured as delayed ratings are closely related to more demanding items within a set of items. Considering advantages of single-item subjective ratings (easy to implement even in huge samples, low time exposure, and suitableness for repeated measures) and disadvantages of alternative ways to measure cognitive load (regarding cost and time efficiency and problem of additional load), current results confirm the use of these items to get an impression of the overall cognitive load. However, the results also suggest that both items do not measure the same thing and researchers should therefore discuss carefully which item they use and how this may limit the results of their study.Keywords
Funding Information
- Deutsche Forschungsgemeinschaft (WA2829/5-1)
- Universität Duisburg-Essen
This publication has 55 references indexed in Scilit:
- Measuring physician cognitive load: validity evidence for a physiologic and a psychometric toolAdvances in Health Sciences Education, 2016
- Timing and Frequency of Mental Effort Measurement: Evidence in Favour of Repeated MeasuresApplied Cognitive Psychology, 2012
- Cognitive Load Theory2011
- Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive LoadEducational Psychology Review, 2010
- Multidimensional Item Response Theory Models2009
- The imagination effect increases with an increased intrinsic cognitive loadApplied Cognitive Psychology, 2007
- A motivational perspective on the relation between mental effort and performance: Optimizing learner involvement in instructionEducational Technology Research and Development, 2005
- Cognitive load and the imagination effectApplied Cognitive Psychology, 2004
- Forschungsmethoden und Evaluation2002
- Multicomponent Response Models1997