Artificial Intelligence in Simulation Design
- 20 August 2026
- journal article
- research article
- Published by Wolters Kluwer Health in Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare
Abstract
Introduction: The creation of simulation scenarios is typically time-consuming and requires a high cognitive load. Artificial intelligence (AI) and large language models (LLMs) can generate text, including simulation scenarios, but the output quality can be very user-dependent. The goal of this study is to explore if a customized version of ChatGPT can create simulation scenarios that are comparable to those written by humans. Methods: In this observational cross-sectional study, simulation content experts scored 8 scenarios covering 4 pediatric emergency medicine (PEM) topics with one version written by health care educators and published on MedEdPORTAL, and another by a custom ChatGPT. Experts were blinded to the source of each case. The scenarios were assessed in 3 ways: an objective score from an established simulation scoring checklist, a subjective quality rating, and whether the experts would recommend the scenario for use. Results: Seven experts with an average of 11 years of PEM training and an average of 10 years of simulation work scored simulations. While all 4 AI-generated cases had lower average scores on both quantitative assessments than their human-written counterparts, these results were not statistically significant. The majority of experts recommended each case to colleagues, regardless of source. There was moderate to excellent intrarater reliability. Conclusion: Human-written cases received higher scores than the AI-written cases. While these scores were not statistically significant, these differences highlight the importance of keeping a human-expert influence when using AI to produce simulation cases. This ChatGPT tool can therefore be useful in assisting in the timely creation of scenarios, allowing facilitators, regardless of their technical skill with AI, to decrease cognitive load. Next steps should move beyond evaluating the scenario text to include evaluation of the experience of facilitating the ChatGPT-derived pediatric simulations as well as assessing the perspectives of simulation operators.Keywords
This publication has 19 references indexed in Scilit:
- Artificial intelligence (AI)-facilitated debriefing: A pilot studyClinical Simulation in Nursing, 2025
- Exploring the Use of a Large Language Model in Simulation Debriefing: An Observational Simulation-Based Pilot StudySimulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 2025
- From Human-in-the-Loop to Human-in-PowerThe American Journal of Bioethics, 2024
- Enhancing Healthcare Education: Leveraging ChatGPT for Innovative Simulation ScenariosClinical Simulation in Nursing, 2023
- Artificial Intelligence and the SimulationistsSimulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 2023
- Advancing Healthcare Simulation Through Artificial Intelligence and Machine Learning: Exploring InnovationsClinical Simulation in Nursing, 2023
- Healthcare Simulation Standards of Best PracticeTM Simulation DesignClinical Simulation in Nursing, 2021
- Development of a Simulation Scenario Evaluation Tool (SSET): modified Delphi studyBMJ Simulation and Technology Enhanced Learning, 2019
- The REDCap consortium: Building an international community of software platform partnersJournal of Biomedical Informatics, 2019
- Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics supportJournal of Biomedical Informatics, 2009