Multicentre sleep‐stage scoring agreement in the Sleep Revolution project
Open Access
- 13 June 2023
- journal article
- research article
- Published by Wiley in Journal of Sleep Research
- Vol. 33 (1) , e13956
Abstract
Summary: Determining sleep stages accurately is an important part of the diagnostic process for numerous sleep disorders. However, as the sleep stage scoring is done manually following visual scoring rules there can be considerable variation in the sleep staging between different scorers. Thus, this study aimed to comprehensively evaluate the inter‐rater agreement in sleep staging. A total of 50 polysomnography recordings were manually scored by 10 independent scorers from seven different sleep centres. We used the 10 scorings to calculate a majority score by taking the sleep stage that was the most scored stage for each epoch. The overall agreement for sleep staging was κ = 0.71 and the mean agreement with the majority score was 0.86. The scorers were in perfect agreement in 48% of all scored epochs. The agreement was highest in rapid eye movement sleep (κ = 0.86) and lowest in N1 sleep (κ = 0.41). The agreement with the majority scoring varied between the scorers from 81% to 91%, with large variations between the scorers in sleep stage‐specific agreements. Scorers from the same sleep centres had the highest pairwise agreements at κ = 0.79, κ = 0.85, and κ = 0.78, while the lowest pairwise agreement between the scorers was κ = 0.58. We also found a moderate negative correlation between sleep staging agreement and the apnea–hypopnea index, as well as the rate of sleep stage transitions. In conclusion, although the overall agreement was high, several areas of low agreement were also found, mainly between non‐rapid eye movement stages.Keywords
Funding Information
- Hengityssairauksien Tutkimussäätiö
- Kuopion Yliopistollinen Sairaala (5041803, 5041794, 5041809)
- Maud Kuistilan Muistosäätiö
- National Health and Medical Research Council (Ideas Grant 2001729, EU‐Horizons Grant 2007001)
- Suomen Kulttuurirahasto
- Suomen Tuberkuloosin Vastustamisyhdistyksen Säätiö
This publication has 19 references indexed in Scilit:
- STOP-Bang QuestionnaireChest, 2016
- Process and outcome for international reliability in sleep scoringSleep and Breathing, 2014
- Agreement in the Scoring of Respiratory Events and Sleep Among International Sleep CentersSLEEP, 2013
- Inter-scorer Reliability between Sleep Centers Can Teach Us What to Improve in the Scoring RulesJournal of Clinical Sleep Medicine, 2013
- The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage ScoringJournal of Clinical Sleep Medicine, 2013
- Prevalence of restless legs syndrome among adults in Iceland and Sweden: Lung function, comorbidity, ferritin, biomarkers and quality of lifeSleep Medicine, 2010
- Interrater reliability for sleep scoring according to the Rechtschaffen & Kales and the new AASM standardJournal of Sleep Research, 2009
- Validation of the International Restless Legs Syndrome Study Group rating scale for restless legs syndromeSleep Medicine, 2003
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960