Towards automated transcribing and coding of embodied teamwork communication through multimodal learning analytics
Open Access
- 30 May 2024
- journal article
- research article
- Published by Wiley in British Journal of Educational Technology
- Vol. 55 (4) , 1673-1702
Abstract
Effective collaboration and teamwork skills are critical in high‐risk sectors, as deficiencies in these areas can result in injuries and risk of death. To foster the growth of these vital skills, immersive learning spaces have been created to simulate real‐world scenarios, enabling students to safely improve their teamwork abilities. In such learning environments, multiple dialogue segments can occur concurrently as students independently organise themselves to tackle tasks in parallel across diverse spatial locations. This complex situation creates challenges for educators in assessing teamwork and for students in reflecting on their performance, especially considering the importance of effective communication in embodied teamwork. To address this, we propose an automated approach for generating teamwork analytics based on spatial and speech data. We illustrate this approach within a dynamic, immersive healthcare learning environment centred on embodied teamwork. Moreover, we evaluated whether the automated approach can produce transcriptions and epistemic networks of spatially distributed dialogue segments with a quality comparable to those generated manually for research objectives. This paper makes two key contributions: (1) it proposes an approach that integrates automated speech recognition and natural language processing techniques to automate the transcription and coding of team communication and generate analytics; and (2) it provides analyses of the errors in outputs generated by those techniques, offering insights for researchers and practitioners involved in the design of similar systems. Practitioner notes: What is currently known about this topic Immersive learning environments simulate real‐world situations, helping students improve their teamwork skills. In these settings, students can have multiple simultaneous conversations while working together on tasks at different physical locations. The dynamic nature of these interactions makes it hard for teachers to assess teamwork and communication and for students to reflect on their performance. What this paper adds We propose a method that employs multimodal learning analytics for automatically generating teamwork‐related insights into the content of student conversations. This data processing method allows for automatically transcribing and coding spatially distributed dialogue segments generated from students working in teams in an immersive learning environment and enables downstream analysis. This approach uses spatial analytics, natural language processing and automated speech recognition techniques. Implications for practitioners Automated coding of dialogue segments among team members can help create analytical tools to assist in evaluating and reflecting on teamwork. By analysing spatial and speech data, it is possible to apply learning analytics advancements to support teaching and learning in fast‐paced physical learning spaces where students can freely engage with one another.Keywords
Funding Information
- Australian Research Council
This publication has 78 references indexed in Scilit:
- Ordered Network Analysis2023
- The Mathematical Foundations of Epistemic Network Analysis2021
- Using Motion Sensors to Understand Collaborative Interactions in Digital Fabrication Labs2020
- Moodoo: Indoor Positioning Analytics for Characterising Classroom Teaching2020
- “Lost in Digitization”: A Spatial Journey in Emergency Response and Pragmatic Legitimacy2019
- When coding-and-counting is not enough: using epistemic network analysis (ENA) to analyze verbal data in CSCL researchInternational Journal of Computer-Supported Collaborative Learning, 2018
- Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learningContemporary Educational Psychology, 2017
- Experience API: Flexible, Decentralized and Activity-Centric Data CollectionTechnology, Knowledge and Learning, 2015
- Team learning: building shared mental modelsInstructional Science, 2010
- A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailabilityJournal of Pharmacokinetics and Biopharmaceutics, 1987