Jessica Andrews-Todd

Collaborative problem solving (CPS) skills gain great attention in recent years as people are often required to work in teams to solve problems in the complex working environment. Even though CPS is usually treated as a domain-general competency in assessment, many CPS tasks may be closely related to the team member's domain knowledge. This research aims to explore and disentangle the relations between CPS skills and domain knowledge. Cluster analysis on 378 individuals from 126 teams solving an online electronics task discovered two types of collaboration patterns, which also differ on the prior knowledge levels. However, these two subgroups were not significantly different on the task performance. This finding uncovered interesting interactions between CPS skills and domain knowledge, and emphasizes the importance to consider the task features and domain knowledge in CPS assessment and learning.

Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals’ CPS competency are necessary, as traditional assessment types such as multiple -choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals’ CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students’ display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts.

Collaboration can support performance on a variety of tasks, but recent projects have indicated that group collaborations can also be associated with memory decrements. For example, when people discuss ideas, any shared inaccurate information can be used by group members to complete subsequent tasks. Across two experiments, we examined whether this social contagion is influenced by performance pressures that regularly emerge during group interactions. In Experiment 1, participants under individual-directed pressure, goal-directed pressure, or control conditions studied word lists before completing a collaborative recall with a confederate partner who occasionally recalled incorrect words. We examined whether partner-produced inaccuracies contaminated participants' memories for the list contents on a subsequent individual recall. Goal-directed pressure, focused on monitoring partner accuracy, facilitated subsequent individual recall, as demonstrated by reduced reproductions of partner-generated inaccuracies. In contrast, individual-directed pressure, focused on appearing competent, resulted in greater use of those inaccuracies. Experiment 2 ruled out that the benefits associated with goal-directed pressure were solely due to warning participants about the possibility of social contagion. These results demonstrate that different instantiations of pressure can help or hinder memory-related performance in collaborative settings. Under the right conditions, pressure can help overcome the effects of exposure to inaccurate information. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Collaborative problem solving (CPS) is an important 21st‐century skill for academic and career success, and as a result, there is increased interest among businesses and educational institutions in the assessment and development of CPS skills. CPS skills are difficult to measure using traditional forms of assessment, and that difficulty has led to the use of computer environments that allow individuals to interact in complex situations and capture all actions throughout the process. In the current paper, we describe the design of a collaborative online mathematics task and explore student perceptions of the task and the extent to which the task elicits CPS skills. Results revealed areas of difficulty for students and showed that the task mostly elicited skills associated with the social dimension of CPS. We describe modifications to the task to provide better opportunities for students to display a broader range of CPS skills. An upcoming larger scale pilot study will explore whether the redesigned task can elicit the less observed CPS skills.

Collaborative problem solving (CPS) has been deemed a critical twenty-first century competency for a variety of contexts. However, less attention has been given to work aimed at the assessment and acquisition of such capabilities. Recently large scale efforts have been devoted toward assessing CPS skills, but there are no agreed upon guiding principles for assessment of this complex construct, particularly for assessment in digital performance situations. There are notable challenges in conceptualizing the complex construct and extracting evidence of CPS skills from large streams of data in digital contexts such as games and simulations. In the current paper, we discuss how the in-task assessment framework (I-TAF), a framework informed by evidence-centered design, can provide guiding principles for the assessment of CPS in these contexts. We give specific attention to one aspect of I-TAF, ontologies, and describe how they can be used to instantiate the student model in evidence-centered design which lays out what we wish to measure in a principled way. We further discuss how ontologies can serve as an anchor representation for other components of assessment such as scoring rubrics, evidence identification, and task design.