Video-Based Neonatal Pain Assessment in Uncontrolled Conditions
- 16 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 28 (1) , 239-250
Abstract
Objective : Neonatal pain can have long-term adverse effects on newborns' cognitive and neurological development. Video-based N eonatal P ain A ssessment (NPA) method has gained increasing attention due to its performance and practicality. However, existing methods focus on assessment under controlled environments while ignoring real-life disturbances present in uncontrolled conditions. Methods : We propose a video-based NPA method, which is robust to four real-life disturbances and adaptively highlights keyframes. Our method involves a region-channel-attention module for extracting facial features under the disturbances of facial occlusion and pose variation; a body language analysis module robust to disturbances from body occlusion and movement interference, which utilizes skeleton sequences to represent the neonate's body; and a keyframes-aware convolution to get rid of information located at non-contributing moments. For evaluation, we built an NPA video dataset of 1091 neonates with disturbance annotations. Results : The results show that our method consistently outperforms state-of-the-art methods on the full dataset and nine subsets, where it achieves an accuracy of 91.04% on the full dataset with an accuracy increment of 6.27%. Contributions : We present the problem of video-based NPA under uncontrolled conditions, propose a method robust to four disturbances, and construct a video NPA dataset, thus facilitating the practical applications of NPA.Keywords
Funding Information
- National Natural Science Foundation of China (62306272)
- Zhejiang Provincial Natural Science Foundation of China (LQ21F010016, LGF20H040008)
- Zhejiang Provincial Key Research and Development Program of China (2021C03027)
This publication has 34 references indexed in Scilit:
- Gate-variants of Gated Recurrent Unit (GRU) neural networks2017
- Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild2017
- Automated Pain Assessment in Neonates2017
- Translational studies identify long-term impact of prior neonatal pain experiencePain, 2017
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- FaceNet: A unified embedding for face recognition and clustering2015
- Too many crying babies: a systematic review of pain management practices during immunizations on YouTubeBMC Pediatrics, 2014
- Relevance Vector Machine Learning for Neonate Pain Intensity Assessment Using Digital ImagingIEEE Transactions on Biomedical Engineering, 2010
- Are There Long-Term Consequences of Pain in Newborn or Very Young Infants?The Journal of Perinatal Education, 2004