What can gaze behaviors, neuroimaging data, and test scores tell us about test method effects and cognitive load in listening assessments?
Open Access
- 3 July 2021
- journal article
- research article
- Published by SAGE Publications in Language Testing
- Vol. 39 (1) , 56-89
Abstract
The aim of this study was to investigate how test methods affect listening test takers’ performance and cognitive load. Test methods were defined and operationalized as while-listening performance (WLP) and post-listening performance (PLP) formats. To achieve the goal of the study, we examined test takers’ (N = 80) brain activity patterns (measured by functional near-infrared spectroscopy (fNIRS)), gaze behaviors (measured by eye-tracking), and listening performance (measured by test scores) across the two test methods. We found that the test takers displayed lower activity levels across brain regions supporting comprehension during the WLP tests relative to the PLP tests. Additionally, the gaze behavioral patterns exhibited during the WLP tests suggested that the test takers adopted keyword matching and “shallow listening.” Together, the neuroimaging and gaze behavioral data indicated that the WLP tests imposed a lower cognitive load on the test takers than the PLP tests. However, the test takers performed better with higher test scores for one of two WLP tests compared with the PLP tests. By incorporating eye-tracking and neuroimaging in this exploration, this study has advanced the current knowledge on cognitive load and the impact imposed by different listening test methods. To advance our knowledge of test validity, other researchers could adopt our research protocol and focus on extending the test method framework used in this study.Keywords
Funding Information
- National Institute of Education (RI 1/18 VSA)
This publication has 46 references indexed in Scilit:
- What is working memory capacity, and how can we measure it?Frontiers in Psychology, 2013
- Left-handedness and language lateralization in childrenBrain Research, 2012
- The Neural Organization of Semantic Control: TMS Evidence for a Distributed Network in Left Inferior Frontal and Posterior Middle Temporal GyrusCerebral Cortex, 2010
- Neural Mechanisms Underlying the Impact of Visual Distraction on Retrieval of Long-Term MemoryJournal of Neuroscience, 2010
- Empiricism versus connoisseurship: Establishing the appropriacy of texts in tests of academic readingLanguage Testing, 2010
- Teaching and Testing Listening Comprehension2009
- A meta-analysis of test format effects on reading and listening test performance: Focus on multiple-choice and open-ended formatsLanguage Testing, 2009
- Ventral and dorsal pathways for languageProceedings of the National Academy of Sciences, 2008
- The extended language network: A meta‐analysis of neuroimaging studies on text comprehensionHuman Brain Mapping, 2007
- Validity and washback in language testingLanguage Testing, 1996