Internet of things assisted deep learning enabled driver drowsiness monitoring and alert system using CNN-LSTM framework
- 26 November 2024
- journal article
- research article
- Published by IOP Publishing in Engineering Research Express
- Vol. 6 (4) , 045239
Abstract
Driver fatigue has been generally viewed as a critical road safety factor and has been cited for contributing to a good percentage of traffic accidents. Developing systems to monitor and alert drowsy drivers is essential to reduce incidents. This research proposes a robust framework for driver drowsiness detection using a CNN-LSTM architecture developed by fusing facial landmark analysis with multiple aspect ratios. It uses, as key metrics in detecting drowsiness, the Eye Aspect Ratio (EAR), Pupil Circularity (PUC), the Mouth Aspect Ratio (MAR), and the Mouth over Eye Aspect Ratio (MOE). CNN-LSTM had been trained on YawDD, NITYMD, FL3D, and custom datasets. Data augmentation techniques such as flipping, scaling, shearing, rotation, brightness, and contrast adjustment are used to generalise under different illumination conditions and driver postures. The system is implemented on NVIDIA’s 128-core Jetson Nano GPU platform and does real-time processing of video frames captured by a CSI camera. It detects eye closure and yawning as symptoms of driver fatigue and immediately raises an alert through vibrations in the seatbelt and pre-recorded voice messages. The Internet connectivity allows remote monitoring via mobile applications, making it safer by ensuring that alerts reach the driver and passengers. This CNN-LSTM model has been carefully tested for various scenarios, including day and night conditions, proving its effectiveness. The proposed framework indicated excellent performance concerning accuracy (98%), precision (95%), recall (93%), F1 score (94%), and AUC (99%), thereby revealing its efficiency in practical scenarios. By incorporating EAR, MAR, PUC, and MOE for the early detection of drowsiness in this CNN-LSTM architecture, the system will be able to alert the driver ahead of time so that necessary precautions can be taken to avoid an accident. The proposed approach enhances the driver’s safety and is a scalable solution that adapts to different environments and populations.Keywords
This publication has 36 references indexed in Scilit:
- A driver stress detection model via data augmentation based on deep convolutional recurrent neural networkExpert Systems with Applications, 2023
- Multimodal driver distraction detection using dual-channel network of CNN and TransformerExpert Systems with Applications, 2023
- Driver drowsiness detection and smart alerting using deep learning and IoTInternet of Things, 2023
- A decomposition-based hybrid ensemble CNN framework for driver fatigue recognitionInformation Sciences, 2023
- LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEGExpert Systems with Applications, 2022
- Driver stress detection via multimodal fusion using attention-based CNN-LSTMExpert Systems with Applications, 2021
- Using long short term memory and convolutional neural networks for driver drowsiness detectionAccident Analysis & Prevention, 2021
- Real-time monitoring of driver drowsiness on mobile platforms using 3D neural networksNeural Computing and Applications, 2019
- Driver Sleepiness Detection Using LSTM Neural Network2018
- Driver drowsiness detection using hybrid convolutional neural network and long short-term memoryMultimedia Tools and Applications, 2018