Temporal and sleep stage‐dependent agreement in manual scoring of respiratory events
Authors:
- Minna PitkänenMPMinna Pitkänen
- Henna PitkänenHPHenna Pitkänen
- Rajdeep Kumar NathRNRajdeep Kumar Nath
- Sami NikkonenSNSami Nikkonen
- Samu KainulainenSKSamu Kainulainen
- Henri KorkalainenHKHenri Korkalainen
- Erna Sif ArnardottirErna Sif Arnardottir
- Sigridur SigurdardottirSSSigridur Sigurdardottir
- Thomas PenzelThomas Penzel
- Francesco FanfullaFFFrancesco Fanfulla
- Ulla AnttalainenUAUlla Anttalainen
Open Access
- 4 November 2024
- journal article
- research article
- Published by Wiley in Journal of Sleep Research
- Vol. 34 (3) , e14391
Abstract
Summary: Obstructive sleep apnea diagnosis is based on the manual scoring of respiratory events. The agreement in the manual scoring of the respiratory events lacks an in‐depth investigation as most of the previous studies reported only the apnea–hypopnea index or overall agreement, and not temporal, second‐by‐second or event subtype agreement. We hypothesized the temporal and subtype agreement to be low because the event duration or subtypes are not generally considered in current clinical practice. The data comprised 50 polysomnography recordings scored by 10 experts. The respiratory event agreement between the scorers was calculated using kappa statistics in a second‐by‐second manner. Obstructive sleep apnea severity categories (no obstructive sleep apnea/mild/moderate/severe) were compared between scorers. The Fleiss' kappa value for binary (event/no event) respiratory event scorings was 0.32. When calculated separately within N1, N2, N3 and R, the Fleiss' kappa values were 0.12, 0.23, 0.22 and 0.23, respectively. Binary analysis conducted separately for the event subtypes showed the highest Fleiss' kappa for hypopneas to be 0.26. In 34% of the participants, the obstructive sleep apnea severity category was the same regardless of the scorer, whereas in the rest of the participants the category changed depending on the scorer. Our findings indicate that the agreement of manual scoring of respiratory events depends on the event type and sleep stage. The manual scoring has discrepancies, and these differences affect the obstructive sleep apnea diagnosis. This is an alarming finding, as ultimately these differences in the scorings affect treatment decisions.Keywords
Funding Information
- Suomen Kulttuurirahasto
- Magnus Ehrnroothin Säätiö
- Sigrid Juséliuksen Säätiö (230216)
- Horizon 2020 Framework Programme (965417)
This publication has 27 references indexed in Scilit:
- External proficiency testing improves inter-scorer reliability of polysomnography scoringSleep and Breathing, 2022
- Addressing database variability in learning from medical data: An ensemble-based approach using convolutional neural networks and a case of study applied to automatic sleep scoringComputers in Biology and Medicine, 2020
- Obstructive sleep apnea phenotypes in men based on characteristics of respiratory events during polysomnographySleep and Breathing, 2019
- Correlates of Obstructive Apnea DurationLung, 2013
- High STOP-Bang score indicates a high probability of obstructive sleep apnoeaBritish Journal of Anaesthesia, 2012
- Validation of the International Restless Legs Syndrome Study Group rating scale for restless legs syndromeSleep Medicine, 2003
- Scoring variability between polysomnography technologists in different sleep laboratoriesSleep Medicine, 2002
- Reliability of Scoring Respiratory Disturbance Indices and Sleep StagingSLEEP, 1998
- Forming inferences about some intraclass correlation coefficients.Psychological Methods, 1996
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971