Inter-rater disagreement in manual scoring of intensive care unit sleep data
Authors:
Open Access
- 1 April 2025
- journal article
- Published by Springer Nature in BMC Research Notes
- Vol. 18 (1) , 138
Abstract
Severe sleep disruption is common among intensive care unit (ICU) patients. However, the applicability of standard sleep scoring guidelines by the American Academy of Sleep Medicine (AASM) has been questioned, with most polysomnography (PSG) studies in critically ill patients reporting difficulties in setting up and processing and scoring the recordings. The present study explores human inter-rater agreement in sleep stage scoring following the AASM guidelines, within a heterogenous ICU patient cohort. Two human experts independently scored a total of 51,454 epochs in 20 PSG recordings acquired at the ICU. Epoch-per-epoch comparison of scored stages revealed a Cohen's κ coefficient of agreement of 0.36 for standard 5-stage scoring. Highest agreement occurred in Wake (κ = 0.46), while REM showed the lowest (κ = 0.12). Significant correlations were found between inter-rater agreement, and Simplified Acute Physiology Score (SAPS II, r = - 0.506, p = 0.038), and 12-month mortality (r = - 0.524, p = 0.031). Comparison with similar studies underscore challenges in applying AASM criteria to ICU patients. Despite accounting for artifacts, disparities persisted, emphasizing the need for a nuanced exploration of factors influencing scoring inconsistencies in critically ill patients. Trial was registered as "Sleep and biorhythm in the ICU", in the Centrale Commissie Mensgebonden Onderzoek register, with number NL-OMON43659 ( https://onderzoekmetmensen.nl/nl/trial/43659 ), on registration date august 4th 2015.Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Delirium in the Intensive Care Unit: Occurrence and Clinical Course in Older PatientsJournal of the American Geriatrics Society, 2003
- Contribution of the Intensive Care Unit Environment to Sleep Disruption in Mechanically Ventilated Patients and Healthy SubjectsAmerican Journal of Respiratory and Critical Care Medicine, 2003
- The effects of total sleep deprivation, selective sleep interruption and sleep recovery on pain tolerance thresholds in healthy subjectsJournal of Sleep Research, 2001
- Abnormal Sleep/Wake Cycles and the Effect of Environmental Noise on Sleep Disruption in the Intensive Care UnitAmerican Journal of Respiratory and Critical Care Medicine, 2001
- Effects of guidelines implementation in a surgical intensive care unit to control nighttime light and noise levelsCritical Care Medicine, 2000
- Sleep in Critically Ill Patients Requiring Mechanical VentilationChest, 2000
- Impact of sleep debt on metabolic and endocrine functionThe Lancet, 1999
- Patient Perception of Sleep Quality and Etiology of Sleep Disruption in the Intensive Care UnitAmerican Journal of Respiratory and Critical Care Medicine, 1999
- Partial night sleep deprivation reduces natural killer and celhdar immune responses in humansThe FASEB Journal, 1996
- The Measurement of Observer Agreement for Categorical DataBiometrics, 1977