Kastoori Kalaivanan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Kastoori Kalaivanan has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Shen-Hsing Annabel Chen
Nanyang Technological University
6 shared publications - Barbara Jacoquelyn Sahakian
Department of Psychiatry, University of Cambridge, Cambridge, UK
4 shared publications - Chew Lee Teo
National Institute of Education, Singapore, Singapore
3 shared publications - Trevor William Robbins
University of Cambridge
3 shared publications - Victoria Leong
Early Mental Potential and Wellbeing Research (EMPOWER) Centre, Nanyang Technological University, Singapore, Singapore
3 shared publications - Chrysoula Vassiliu
University of Cambridge, UK
2 shared publications
Latest Publications
- Open Access
- 25 June 2026
- Published by MDPI AG in Journal of Intelligence
- Vol. 14  (7) , 117
Task Engagement in Matrix Reasoning Performance: A Cross-Cultural Investigation in China and the United Kingdom
Matrix reasoning tasks remain among the most widely used instruments for assessing abstract reasoning and are often assumed to be culturally neutral. However, this assumption has been challenged by studies reporting significant cross-cultural variation in performance on nonverbal matrix reasoning tasks, even when groups show comparable performance on verbal measures of general cognitive ability. One plausible reason is that many matrix reasoning tasks rely primarily on accuracy-based performance metrics while providing limited insight into response timing and task engagement during problem solving. The present study examined the Matrix Reasoning Item Bank (MaRs-IB), a new online matrix reasoning instrument integrating both accuracy and response time, in 458 participants from China and the UK. Results demonstrated strong psychometric properties across both cultural contexts, while also revealing systematic between-group differences in overall task performance. Chinese participants were generally slower but more accurate, whereas UK participants responded more quickly with lower overall accuracy. Rather than reflecting a classical speed–accuracy trade-off, these patterns may indicate cross-cultural variation in persistence, deliberative engagement, and the metacognitive regulation of cognitive effort during reasoning tasks. In particular, Chinese participants allocated more time before responding and persisted longer on challenging task items, whereas UK participants demonstrated relatively faster responding and shorter response times on more challenging items. These findings suggest that cross-cultural differences in matrix reasoning performance may reflect not only differences in observed performance levels, but also variation in how participants allocate time and sustain engagement during cognitively demanding tasks.
- 25 June 2026
- Published by MDPI AG in Journal of Intelligence
- Vol. 14  (7) , 117
Open AccessTask Engagement in Matrix Reasoning Performance: A Cross-Cultural Investigation in China and the United Kingdom
Matrix reasoning tasks remain among the most widely used instruments for assessing abstract reasoning and are often assumed to be culturally neutral. However, this assumption has been challenged by studies reporting significant cross-cultural variation in performance on nonverbal matrix reasoning tasks, even when groups show comparable performance on verbal measures of general cognitive ability. One plausible reason is that many matrix reasoning tasks rely primarily on accuracy-based performance metrics while providing limited insight into response timing and task engagement during problem solving. The present study examined the Matrix Reasoning Item Bank (MaRs-IB), a new online matrix reasoning instrument integrating both accuracy and response time, in 458 participants from China and the UK. Results demonstrated strong psychometric properties across both cultural contexts, while also revealing systematic between-group differences in overall task performance. Chinese participants were generally slower but more accurate, whereas UK participants responded more quickly with lower overall accuracy. Rather than reflecting a classical speed–accuracy trade-off, these patterns may indicate cross-cultural variation in persistence, deliberative engagement, and the metacognitive regulation of cognitive effort during reasoning tasks. In particular, Chinese participants allocated more time before responding and persisted longer on challenging task items, whereas UK participants demonstrated relatively faster responding and shorter response times on more challenging items. These findings suggest that cross-cultural differences in matrix reasoning performance may reflect not only differences in observed performance levels, but also variation in how participants allocate time and sustain engagement during cognitively demanding tasks.
- Open Access
- 5 June 2026
- Published by Springer Nature in BMC Psychology
- Vol. 14  (1)
The association between cognitive flexibility and creativity across childhood and adolescence: a scoping review
Cognitive flexibility (CF), the ability to adaptively shift between perspectives, ideas, or strategies, is increasingly recognised as an important executive function underlying creativity. Creativity broadly refers to the capacity to generate ideas or solutions that are both unique and suitable to a given context. A positive association between CF and creativity is well-documented in adults. However, evidence from childhood and adolescence, which are developmental periods characterised by heightened neuroplasticity and the maturation of executive control, remains fragmented. This is largely due to substantial methodological heterogeneity across studies. This scoping review synthesises empirical studies published in the past decade that examine the association between CF and creativity in individuals aged 0-18 years. Specifically, it examines: (1) how CF and creativity have been conceptualised and measured in this age group; (2) whether there is an association between CF and creativity; and (3) whether this association varies across preschoolers, school-going children, and adolescents. Fourteen studies met the inclusion criteria for the scoping review. Definitions and measurement approaches varied widely, spanning performance-based tasks and self‑report instruments. Nine of the studies found a significant positive association between CF and creativity, particularly when CF was assessed as a complex, multifaceted construct and when creativity was assessed by traditional tools. In contrast, five studies reported null or inconsistent findings; these studies often employed narrow task-switching paradigms or creativity tasks incorporating other cognitive demands (e.g., reasoning). Importantly, no consistent age‑related differences were identified in the CF-creativity association. The results combined suggest that creativity may rely more on higher-level, strategic forms of CF that support the integration and exploration of ideas, rather than on simple rule-switching ability. By consolidating existing evidence and highlighting key conceptual and methodological gaps, this review offers a clear direction for future research to take when investigating the association between CF and creativity across developmental stages.
- 5 June 2026
- Published by Springer Nature in BMC Psychology
- Vol. 14  (1)
Open AccessThe association between cognitive flexibility and creativity across childhood and adolescence: a scoping review
Cognitive flexibility (CF), the ability to adaptively shift between perspectives, ideas, or strategies, is increasingly recognised as an important executive function underlying creativity. Creativity broadly refers to the capacity to generate ideas or solutions that are both unique and suitable to a given context. A positive association between CF and creativity is well-documented in adults. However, evidence from childhood and adolescence, which are developmental periods characterised by heightened neuroplasticity and the maturation of executive control, remains fragmented. This is largely due to substantial methodological heterogeneity across studies. This scoping review synthesises empirical studies published in the past decade that examine the association between CF and creativity in individuals aged 0-18 years. Specifically, it examines: (1) how CF and creativity have been conceptualised and measured in this age group; (2) whether there is an association between CF and creativity; and (3) whether this association varies across preschoolers, school-going children, and adolescents. Fourteen studies met the inclusion criteria for the scoping review. Definitions and measurement approaches varied widely, spanning performance-based tasks and self‑report instruments. Nine of the studies found a significant positive association between CF and creativity, particularly when CF was assessed as a complex, multifaceted construct and when creativity was assessed by traditional tools. In contrast, five studies reported null or inconsistent findings; these studies often employed narrow task-switching paradigms or creativity tasks incorporating other cognitive demands (e.g., reasoning). Importantly, no consistent age‑related differences were identified in the CF-creativity association. The results combined suggest that creativity may rely more on higher-level, strategic forms of CF that support the integration and exploration of ideas, rather than on simple rule-switching ability. By consolidating existing evidence and highlighting key conceptual and methodological gaps, this review offers a clear direction for future research to take when investigating the association between CF and creativity across developmental stages.
- Open Access
- 28 April 2026
- Published by Springer Nature in BMC Psychology
- Vol. 14  (1)
Set-shifting and task-switching make differential contributions to divergent thinking in adolescence
Background: Cognitive flexibility, as the capacity to adaptively shift between mental sets, comprises multiple subprocesses, including set-shifting and task-switching, and goes through changes in capacity throughout development. Divergent thinking, a core aspect of human creativity, also goes through substantial development during adolescence, a period marked by rapid refinement of executive functions. Despite their developmental interdependence, the distinct contributions of cognitive flexibility subcomponents to divergent thinking during adolescence remain insufficiently understood. Methods: We examined cognitive flexibility and divergent thinking in a large sample of adolescents ( N = 344). Participants completed established cognitive flexibility measures (e.g., Wisconsin Card Sorting Task; Task-Set Switching), additional cognitive tasks (i.e., working memory, inhibitory control and abstract reasoning) and the Alternate Uses Task to assess fluency and originality in divergent thinking. Results: Findings provide novel evidence that cognitive flexibility subprocesses differentially contribute to divergent thinking. Specifically, set-shifting, as indexed by Wisconsin Card Sorting Task performance, was a positive predictor of both fluency (i.e., number of ideas generated) and originality (i.e., creative quality of those ideas). In contrast, task-switching performance predicted lower originality, suggesting that rapid, cue-driven shifts may impede the deeper idea exploration required for producing novel responses. This dissociation suggest that cognitive flexibility is not a unitary construct; rather, its distinct components exert separable and, at times, opposing influences on divergent thinking. Conclusions: These results underscore the importance of distinguishing cognitive flexibility subprocesses when understanding the cognitive mechanisms underlying creativity. The findings carry educational implications, suggesting that fostering strategic, self-guided flexibility, while tempering overly rapid shifting, may support adolescents’ capacity to generate original solutions to complex problems.
- 28 April 2026
- Published by Springer Nature in BMC Psychology
- Vol. 14  (1)
Open AccessSet-shifting and task-switching make differential contributions to divergent thinking in adolescence
Background: Cognitive flexibility, as the capacity to adaptively shift between mental sets, comprises multiple subprocesses, including set-shifting and task-switching, and goes through changes in capacity throughout development. Divergent thinking, a core aspect of human creativity, also goes through substantial development during adolescence, a period marked by rapid refinement of executive functions. Despite their developmental interdependence, the distinct contributions of cognitive flexibility subcomponents to divergent thinking during adolescence remain insufficiently understood. Methods: We examined cognitive flexibility and divergent thinking in a large sample of adolescents ( N = 344). Participants completed established cognitive flexibility measures (e.g., Wisconsin Card Sorting Task; Task-Set Switching), additional cognitive tasks (i.e., working memory, inhibitory control and abstract reasoning) and the Alternate Uses Task to assess fluency and originality in divergent thinking. Results: Findings provide novel evidence that cognitive flexibility subprocesses differentially contribute to divergent thinking. Specifically, set-shifting, as indexed by Wisconsin Card Sorting Task performance, was a positive predictor of both fluency (i.e., number of ideas generated) and originality (i.e., creative quality of those ideas). In contrast, task-switching performance predicted lower originality, suggesting that rapid, cue-driven shifts may impede the deeper idea exploration required for producing novel responses. This dissociation suggest that cognitive flexibility is not a unitary construct; rather, its distinct components exert separable and, at times, opposing influences on divergent thinking. Conclusions: These results underscore the importance of distinguishing cognitive flexibility subprocesses when understanding the cognitive mechanisms underlying creativity. The findings carry educational implications, suggesting that fostering strategic, self-guided flexibility, while tempering overly rapid shifting, may support adolescents’ capacity to generate original solutions to complex problems.
- Open Access
- 5 December 2025
- Published by Center for Open Science in PsyArXiv
Set-Shifting and Task-Switching Make Differential Contributions to Divergent Thinking in Adolescence
Background: Cognitive flexibility, as the capacity to adaptively shift between mental sets, comprises multiple subprocesses, including set-shifting and task-switching. Divergent thinking, a core aspect of human creativity, also undergoes substantial development during adolescence, a period marked by rapid refinement of executive functions. Despite their developmental interdependence, the distinct contributions of cognitive flexibility subcomponents to divergent thinking during adolescence remain insufficiently understood.Methods: We examined cognitive flexibility and divergent thinking in a large sample of adolescents (N = 344). Participants completed established cognitive flexibility measures (e.g., Wisconsin Card Sorting Task; Task-Set Switching), additional cognitive tasks (i.e., working memory, inhibitory control and abstract reasoning) and the Alternate Uses Task to assess fluency and originality in divergent thinking.Results: Findings provide novel evidence that specific flexibility subprocesses differentially contribute to divergent thinking. Specifically, set-shifting, as indexed by Wisconsin Card Sorting Task performance, was a positive predictor of both fluency (i.e., number of ideas generated) and originality (i.e., creative quality of those ideas). In contrast, effective task-switching performance predicted lower originality, suggesting that rapid, cue-driven shifts may impede the deeper idea exploration required for producing novel responses. This dissociation suggest that cognitive flexibility is not a unitary construct; rather, its distinct components exert separable and, at times, opposing influences on divergent thinking.Conclusions: These results underscore the importance of distinguishing cognitive flexibility subprocesses when understanding the cognitive mechanisms underlying creativity. The findings carry educational implications, suggesting that fostering strategic, self-guided flexibility, while tempering overly rapid shifting, may support adolescents’ capacity to generate original solutions to complex problems.
- 5 December 2025
- Published by Center for Open Science in PsyArXiv
Open AccessSet-Shifting and Task-Switching Make Differential Contributions to Divergent Thinking in Adolescence
Background: Cognitive flexibility, as the capacity to adaptively shift between mental sets, comprises multiple subprocesses, including set-shifting and task-switching. Divergent thinking, a core aspect of human creativity, also undergoes substantial development during adolescence, a period marked by rapid refinement of executive functions. Despite their developmental interdependence, the distinct contributions of cognitive flexibility subcomponents to divergent thinking during adolescence remain insufficiently understood.Methods: We examined cognitive flexibility and divergent thinking in a large sample of adolescents (N = 344). Participants completed established cognitive flexibility measures (e.g., Wisconsin Card Sorting Task; Task-Set Switching), additional cognitive tasks (i.e., working memory, inhibitory control and abstract reasoning) and the Alternate Uses Task to assess fluency and originality in divergent thinking.Results: Findings provide novel evidence that specific flexibility subprocesses differentially contribute to divergent thinking. Specifically, set-shifting, as indexed by Wisconsin Card Sorting Task performance, was a positive predictor of both fluency (i.e., number of ideas generated) and originality (i.e., creative quality of those ideas). In contrast, effective task-switching performance predicted lower originality, suggesting that rapid, cue-driven shifts may impede the deeper idea exploration required for producing novel responses. This dissociation suggest that cognitive flexibility is not a unitary construct; rather, its distinct components exert separable and, at times, opposing influences on divergent thinking.Conclusions: These results underscore the importance of distinguishing cognitive flexibility subprocesses when understanding the cognitive mechanisms underlying creativity. The findings carry educational implications, suggesting that fostering strategic, self-guided flexibility, while tempering overly rapid shifting, may support adolescents’ capacity to generate original solutions to complex problems.
- 25 November 2023
- Published by SAGE Publications in Quarterly Journal of Experimental Psychology
- Vol. 77  (10) , 2023-2039
Lexical tone perception and learning in older adults: A review and future directions
While the literature is well represented in accounting for how aging influences segmental properties of speech, less is known about its influences on suprasegmental properties such as lexical tones. In addition, foreign language learning is increasingly endorsed as being a potential intervention to boost cognitive reserve and overall well-being in older adults. Empirical studies on young learners learning lexical tones are aplenty in comparison with older learners. Challenges in this domain for older learners might be different due to aging and other learner-internal factors. This review consolidates behavioural and neuroscientific research related to lexical tone, speech perception, factors characterising learner groups, and other variables that would influence lexical tone perception and learning in older adults. Factors commonly identified to influence tone learning in younger adult populations, such as musical experience, language background, and motivation in learning a new language, are discussed in relation to older learner groups and recommendations to boost lexical tone learning in older age are provided based on existing studies.
- 25 November 2023
- Published by SAGE Publications in Quarterly Journal of Experimental Psychology
- Vol. 77  (10) , 2023-2039
Lexical tone perception and learning in older adults: A review and future directions
While the literature is well represented in accounting for how aging influences segmental properties of speech, less is known about its influences on suprasegmental properties such as lexical tones. In addition, foreign language learning is increasingly endorsed as being a potential intervention to boost cognitive reserve and overall well-being in older adults. Empirical studies on young learners learning lexical tones are aplenty in comparison with older learners. Challenges in this domain for older learners might be different due to aging and other learner-internal factors. This review consolidates behavioural and neuroscientific research related to lexical tone, speech perception, factors characterising learner groups, and other variables that would influence lexical tone perception and learning in older adults. Factors commonly identified to influence tone learning in younger adult populations, such as musical experience, language background, and motivation in learning a new language, are discussed in relation to older learner groups and recommendations to boost lexical tone learning in older age are provided based on existing studies.
- 7 March 2023
- Published by American Speech Language Hearing Association in Journal of Speech, Language, and Hearing Research
- Vol. 66  (3) , 987-1017
Native Language Perceptual Sensitivity Predicts Nonnative Speech Perception Differently in Younger and Older Singaporean Bilinguals
Purpose: We investigate in this study how individual variability in native language speech perception (termed Perceptual Sensitivity ) influences nonnative speech perception in Singaporean Tamil–English bilinguals. Further, we assess if and how contextual and demographic factors influence Perceptual Sensitivity in the acquired languages and if the influence of Perceptual Sensitivity on nonnative speech perception is different across younger and older bilinguals. Method: Perceptual Sensitivity in the native languages was examined by implementing Tamil and English gating tasks in 87 Singaporean Tamil–English bilinguals from two age groups (younger: 19–33 years; older: 55–70 years). Mandarin lexical tone discrimination was implemented as a measure of nonnative speech perception. Results: There was a wide range of variability in Perceptual Sensitivity scores in both languages across both age groups. Perceptual Sensitivity in the first native language (L1 Tamil) was a robust predictor of nonnative speech perception across both age groups, especially for the older bilinguals. However, general intelligence emerged as a stronger predictor than Tamil Perceptual Sensitivity in younger bilinguals. The influence of Tamil Perceptual Sensitivity on lexical tone perception was not tone-specific, supporting a general enhancement of lexical tone perception with better Tamil Perceptual Sensitivity. There was an influence of demographic factors on English Perceptual Sensitivity in older bilinguals, but not for Tamil and not in younger bilinguals. Conclusions: Our findings corroborate with previous studies in showing that native language Perceptual Sensitivity is positively associated with and predicts nonnative speech perception in younger and older adulthood regardless of language similarity but to varying degrees. Specifically, the influence of Perceptual Sensitivity on nonnative speech perception is stronger in older adulthood, suggesting a possible shift in reliance on crystallized language knowledge with age. Proficiency and use, among other demographic and language variables, do not appear to influence L1 Perceptual Sensitivity in a lesser used language (Tamil) as significantly as previously assumed.
- 7 March 2023
- Published by American Speech Language Hearing Association in Journal of Speech, Language, and Hearing Research
- Vol. 66  (3) , 987-1017
Native Language Perceptual Sensitivity Predicts Nonnative Speech Perception Differently in Younger and Older Singaporean Bilinguals
Purpose: We investigate in this study how individual variability in native language speech perception (termed Perceptual Sensitivity ) influences nonnative speech perception in Singaporean Tamil–English bilinguals. Further, we assess if and how contextual and demographic factors influence Perceptual Sensitivity in the acquired languages and if the influence of Perceptual Sensitivity on nonnative speech perception is different across younger and older bilinguals. Method: Perceptual Sensitivity in the native languages was examined by implementing Tamil and English gating tasks in 87 Singaporean Tamil–English bilinguals from two age groups (younger: 19–33 years; older: 55–70 years). Mandarin lexical tone discrimination was implemented as a measure of nonnative speech perception. Results: There was a wide range of variability in Perceptual Sensitivity scores in both languages across both age groups. Perceptual Sensitivity in the first native language (L1 Tamil) was a robust predictor of nonnative speech perception across both age groups, especially for the older bilinguals. However, general intelligence emerged as a stronger predictor than Tamil Perceptual Sensitivity in younger bilinguals. The influence of Tamil Perceptual Sensitivity on lexical tone perception was not tone-specific, supporting a general enhancement of lexical tone perception with better Tamil Perceptual Sensitivity. There was an influence of demographic factors on English Perceptual Sensitivity in older bilinguals, but not for Tamil and not in younger bilinguals. Conclusions: Our findings corroborate with previous studies in showing that native language Perceptual Sensitivity is positively associated with and predicts nonnative speech perception in younger and older adulthood regardless of language similarity but to varying degrees. Specifically, the influence of Perceptual Sensitivity on nonnative speech perception is stronger in older adulthood, suggesting a possible shift in reliance on crystallized language knowledge with age. Proficiency and use, among other demographic and language variables, do not appear to influence L1 Perceptual Sensitivity in a lesser used language (Tamil) as significantly as previously assumed.
- Open Access
- 6 January 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24  (1) , e28368
A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study
Background: The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. Objective: This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). Methods: A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. Results: The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. Conclusions: These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.
- 6 January 2022
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 24  (1) , e28368
Open AccessA New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study
Background: The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. Objective: This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). Methods: A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. Results: The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. Conclusions: These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.
- Open Access
- 6 March 2021
- Published by JMIR Publications Inc. in JMIR Preprints
A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study (Preprint)
BACKGROUND The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. OBJECTIVE This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). METHODS A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. RESULTS The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. CONCLUSIONS These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.
- 6 March 2021
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessA New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study (Preprint)
BACKGROUND The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. OBJECTIVE This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). METHODS A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. RESULTS The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. CONCLUSIONS These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.