Interrater reliability and agreement of the NEUMOBACT checklist about infection-prevention performance of intensive care nurses in simulation-based scenarios

Authors:

Abstract
Objective: To analyse the interrater reliability of the NEUMOBACT checklist and verify whether consistent results are reproducible. Methods: A validation study with a cross-sectional design, compliant with the GRRAS checklist, among ICU nurses attending a SIMULAZERO course with an Objective Structured Clinical Evaluation simulation format, to verify transfer from theory to clinical practice of knowledge and skills in ventilator-associated pneumonia (VAP) and catheter-related bacteraemia (CRB) prevention. A minimum sample size of 111 pairs of nurse raters was calculated. Interrater agreement was analysed using Gwet’s AC1 for each item and as a total for each of the three checklists in the NEUMOBACT instrument. Results: A total of 95 pairs of valid NEUMOBACT checklists were completed by 190 raters with a median age of 29 [25–35] years, 93.7% were female. At the Central Venous Catheter insertion station, Gwet’s AC1 was 0.934 (95% CI [0.919–0.949]). Only 2 of the 17 items scored below 0.9. At the Endotracheal Suctioning station, Gwet’s AC1 was 0.869 (95% CI [0.851–0.886]). Of the 26 items that made up this station, 16 had an agreement percentage above 0.9, a further 9 were between 0.821 and 0.884, and item 13 had a value of 0.789. At the Patient Care station, Gwet’s AC1 was 0.911 (95% CI [0.896–0.927]). Of the 21 items, 17 showed an agreement percentage above 0.9 and 4 were between 0.810 and 0.894. Conclusions: The interrater reliability of the NEUMOBACT checklist shows substantial agreement between pairs of raters and is therefore validated in this large sample of ICU nurses. Relevance to clinical practice: The NEUMOBACT checklist can be useful for assessing skills before and after training in VAP and CRB prevention measures and during debriefing (post-simulation feedback), to reinforce the scientific evidence behind actions and decisions for items that have been performed incorrectly, thus consolidating training already received.
Funding Information
  • SESSEP Research Grant; https://www.sessep.com/becas-sessep/ (Research Grant 2021)

This publication has 27 references indexed in Scilit: