AutoCLC: Towards Automated Assessment and Feedback on Closed-Loop Communication in Team-based Healthcare Simulation Training
Authors:
Open Access
- 14 January 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 7 (1) , 1-31
Abstract
This study introduces AutoCLC, an AI-powered system designed to assess and provide feedback on Closed-Loop Communication (CLC) in professional learning environments. CLC, where a sender’s Call-Out statement is acknowledged by the receiver’s Check-Back statement, is a critical safety protocol in high-reliability domains, including emergency medicine resuscitation teams. Existing methods for evaluating CLC lack quantifiable metrics and depend heavily on human observation. AutoCLC addresses these limitations by leveraging natural language processing and large language models to analyze audio recordings from Advanced Cardiovascular Life Support (ACLS) simulation training. The system identifies CLC instances, measures their frequency and rate per minute, and categorizes communications as effective, incomplete, or missed. Technical evaluations demonstrate that AutoCLC achieves 78.9% precision for identifying Call-Outs and 74.3% for Check-Backs, with a performance gap of only 5% compared to human annotations. A user study involving 11 cardiac arrest instructors across three training sites supported the need for automated CLC assessment. Instructors found AutoCLC reports valuable for quantifying CLC frequency and quality, as well as for providing actionable, example-based feedback. Participants rated AutoCLC highly, with a System Usability Scale score of 76.4%, reflecting above-average usability. This work represents a significant step toward developing scalable, data-driven feedback systems that enhance individual skills and team performance in high-reliability settings.Keywords
Funding Information
- US National Science Foundation (IIS-2202451, DRL-2506865)
This publication has 65 references indexed in Scilit:
- Virtual Reality for Cardiopulmonary Resuscitation Healthcare Professionals Training: A Systematic ReviewJournal of Medical Systems, 2024
- Human-in-the-loop in artificial intelligence in education: A review and entity-relationship (ER) analysisComputers in Human Behavior: Artificial Humans, 2024
- A systematic review of the role of learning analytics in enhancing feedback practices in higher educationEducational Research Review, 2022
- Explainable Artificial Intelligence in educationComputers and Education: Artificial Intelligence, 2022
- Closed loop communication in the trauma bay: identifying opportunities for team performance improvement through a video review analysisCanadian Journal of Emergency Medicine, 2022
- Automatic detection of potentially ineffective verbal communication for training through simulation in neonatologyEducation and Information Technologies, 2022
- A Meta-Analysis of Task and Training Characteristics that Contribute to or Attenuate the Effectiveness of the After-Action Review (or Debrief)Journal of Business and Psychology, 2022
- The effects of visualization on judgment and decision-making: a systematic literature reviewManagement Review Quarterly, 2021
- Safety gaps in medical team communication: Results of quality improvement efforts in a cardiac catheterization laboratoryCatheterization and Cardiovascular Interventions, 2019
- Resuscitation After Cardiac ArrestJAMA, 2002