Validation of ‘Somnivore’, a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data
Open Access
- 18 March 2019
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 13, 207
Abstract
Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, SomnivoreTM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.Keywords
Funding Information
- Ministero dell’Istruzione, dell’Università e della Ricerca
- Universität Zürich
- Wellcome Trust
- National Institute for Health Research (A90305, A92181)
- National Health and Medical Research Council
- Australasian Sleep Association
This publication has 63 references indexed in Scilit:
- FASTER: an unsupervised fully automated sleep staging method for miceGenes to Cells, 2013
- The American Academy of Sleep Medicine Inter-scorer Reliability Program: Sleep Stage ScoringJournal of Clinical Sleep Medicine, 2013
- Rules for Scoring Respiratory Events in Sleep: Update of the 2007 AASM Manual for the Scoring of Sleep and Associated EventsJournal of Clinical Sleep Medicine, 2012
- Local sleep homeostasis in the avian brain: convergence of sleep function in mammals and birds?Proceedings Of The Royal Society B-Biological Sciences, 2011
- The effect of mindfulness-based therapy on anxiety and depression: A meta-analytic review.Journal of Consulting and Clinical Psychology, 2010
- Open-source logic-based automated sleep scoring software using electrophysiological recordings in ratsJournal of Neuroscience Methods, 2009
- History and future of comparative analyses in sleep researchNeuroscience & Biobehavioral Reviews, 2009
- Sleep Modulates Hypertension in Leptin-Deficient Obese MiceHypertension, 2009
- Interhemispheric asynchrony of the sleep EEG in northern fur sealsCellular and Molecular Life Sciences, 1985
- Circadian rhythms ofMacaca mulatta: Sleep, EEG, body and eye movement, and temperaturePrimates, 1972