Autonomous AI Surveillance: Multimodal Deep Learning for Cognitive and Behavioral Monitoring
Authors:
- M. Abdullah AsadMAM. Abdullah Asad
- Muhammad NaeemMNMuhammad Naeem
Preprint
Abstract
This study presents a novel classroom surveillance system that integrates multiple modalities, including drowsiness, tracking of mobile phone usage, and face recognition,to assess student attentiveness with enhanced precision.The system leverages the YOLOv8 model to detect both mobile phone and sleep usage,(Ghatge et al., 2024) while facial recognition is achieved through LResNet Occ FC body tracking using YOLO and MTCNN.(Durai et al., 2024) These models work in synergy to provide comprehensive, real-time monitoring, offering insights into student engagement and behavior.(S et al., 2023) The framework is trained on specialized datasets, such as the RMFD dataset for face recognition and a Roboflow dataset for mobile phone detection. The extensive evaluation of the system shows promising results. Sleep detection achieves 97. 42% mAP@50, face recognition achieves 86. 45% validation accuracy and mobile phone detection reach 85. 89% mAP@50. The system is implemented within a core PHP web application and utilizes ESP32-CAM hardware for seamless data capture.(Neto et al., 2024) This integrated approach not only enhances classroom monitoring, but also ensures automatic attendance recording via face recognition as students remain seated in the classroom, offering scalability for diverse educational environments.(Banada,2025)Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- AI Powered Attendance System for Class Rooms using Face RecognitionInternational Journal of Scientific Research in Engineering and Management, 2025
- Student Engagement Recognition: Comprehensive Analysis Through EEG and Verification by Image Traits Using Deep Learning TechniquesIEEE Access, 2025
- Classroom Behavior Recognition Based on YOLOX Improved Feature Redundancy Removal2024
- Enhancing Classroom Engagement using Real-Time Student Attention Monitoring with YOLOv92024
- Real Time Facial Recognition-Based Criminal Identification Using MTCNN2024
- Edumetrics: A Revolutionary Step in EducationInternational Journal of Advanced Research in Science, Communication and Technology, 2024
- Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition2024
- Monitoring and improving student attention using deep learning and wireless sensor networksSensors and Actuators A: Physical, 2024
- Systematic Review on Real-Time Students Behavior Monitoring using Machine Learning2023
- Relationship between students’ online learning behavior and course performance: What contextual information matters?Physical Review Physics Education Research, 2020