Abstract
Collaborative problem-solving (CPS) is an essential 21st-century competency, yet its valid and ecologically sound assessment remains challenging due to inconsistent frameworks, poorly aligned task design, and over-reliance on behavior frequency counting in scoring. This study developed and provided empirical support for a theory-driven game-based CPS assessment embedded in a real-time strategy game, using a customized Collaborative Siege Task with asymmetric roles and ill-structured problems, to elicit more realistic collaborative interactions. Based on a validated three-facet CPS framework from Chinese cultural context, the task systematically triggers social and cognitive processes of collaboration across five progressive stages. By sampling from the eastern provinces of China, a pilot study with 36 participants confirmed task feasibility and indicator effectiveness, while a formal study with 298 valid participants examined psychometric properties using item response theory and confirmatory factor analysis. Findings supported the hierarchical higher-order factor structure, satisfactory item difficulty, discrimination, reliability, and criterion-related validity against peer assessment, reasoning ability, and personality. The assessment showed no sex, major, or role bias. This study provides a reliable gamified assessment tool for CPS measurement with demonstrated potential for scalability pending further development of automated scoring procedures, offering implications for large-scale evaluation and competency-oriented instruction.
Funding Information
  • 2025 Youth Fund Project for Humanities and Social Sciences Research of the Ministry of Education (25YJC880029)
  • Yangzhou University (5005/142080184)
  • Yangzhou University (5005/136020503)
  • Green Yang Jinfeng Plan of Yangzhou excellent doctoral funding project (5005/137013654)

This publication has 22 references indexed in Scilit: