A Human-in-the-Loop Method for Annotation of Events in Biomedical Signals
- 13 September 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (1) , 95-106
Abstract
Objective: Building large-scale data bases of biomedical signal recordings for training artificial-intelligence systems involves substantial human effort in data processing and annotation. In the case of event detection, experts need to exhaustively scroll through the recordings and highlight events of interest. Methods: We propose an iterative annotation support algorithm with a human in the loop to improve the efficiency of the annotation process. Our algorithm generates proposal events based on an event detection model trained on incomplete annotations. The human only needs to verify candidate events proposed by the tool instead of scrolling through the entire data set. Our algorithm iterates between proposal generation and verification to leverage the human-in-the-loop feedback to obtain a growing set of event annotations. Results: Our algorithm finds a substantial amount of events at a fraction of the human time spent when comparing with a benchmark method and the normal manual process, finding all events in one data set and 70% of events in another with the human-in-the-loop only viewing 20% of the data. Conclusion: Our results show that combining human and computer effort can substantially speed up the annotation process for events in biomedical signal processing. Significance: Due to its simplicity and minimal reliance on task-specific information, our algorithm is broadly applicable, unlocking substantial improvements in the scalability and efficiency of biomedical signal annotation.Keywords
Funding Information
- European Research Council (802895)
- Vlaamse regering
This publication has 48 references indexed in Scilit:
- Automated detection of artefacts in neonatal EEG with residual neural networksComputer Methods and Programs in Biomedicine, 2021
- Artefact detection and quality assessment of ambulatory ECG signalsComputer Methods and Programs in Biomedicine, 2019
- A survey on semi-supervised learningMachine Learning, 2019
- Machine learning in the electrocardiogramJournal of Electrocardiology, 2019
- Attention gated networks: Learning to leverage salient regions in medical imagesMedical Image Analysis, 2019
- Evaluation of Interactive Machine Learning Systems2018
- Incremental on-line learning: A review and comparison of state of the art algorithmsNeurocomputing, 2018
- EEG interpretation reliability and interpreter confidence: A large single-center studyEpilepsy & Behavior, 2014
- Incorporating structural information from the multichannel EEG improves patient-specific seizure detectionClinical Neurophysiology, 2012
- Application of independent component analysis in removing artefacts from the electrocardiogramNeural Computing and Applications, 2005