Shyam Sunder Prasad

Industry 5.0 and increased industrial automation have driven the demand for systems recognizing human activities in industrial environments. Vision-based systems for human activity recognition at industrial sites may be helpful in ergonomic studies. Besides, these systems may help identify possible deviations in assembly line standard operating procedures (SOPs) and facilitate early rejection of items. The primary challenge these systems face is the limited research dedicated to accurately comprehending and interpreting human actions and intentions within the intricate and dynamic contexts of industrial settings. While human action recognition (HAR) has seen significant exploration in machine learning, its application within industrial settings remains relatively unexplored. Besides, the current research lacks a realistic open-source dataset accurately representing human actions in industrial settings. To this end, this article first introduces the Lathe-operator activity monitoring in industrial surroundings (LAMIS) database, a specialized database that covers 17 categories of industrial-like actions. Second, the article presents the design and implementation of a novel deep learning architecture called industrial action recognition network (IAR-Net) for industrial activity recognition. IAR-Net uses RGB spatiotemporal cues to capture human context and granularities. We trained the proposed IAR-Net using a novel adaptive frame sampling approach that adaptively selects keyframes from video clips, reducing the overall computational cost. The IAR-Net model achieves a baseline recognition accuracy of 85.39% on the in-house LAMIS database. In addition, the model delivered a state-of-the-art accuracy of 95.23% on the related benchmark HRI30 dataset. The results demonstrate the efficacy of the proposed IAR-Net model and its adaptivity in several human activity recognition tasks in industrial settings.

Indoor surveillance is crucial for ensuring the safety and security of occupants within the premises. Only those who are ill or elderly tend to spend the most time at home. The use of indoor surveillance to continuously monitor these people’s security could help in the early detection and avoidance of tragic incidents. Ensuring privacy while achieving this task has led to a recent research focus on protecting privacy in human fall detection. This paper attempts to address the issue of privacy-preserving fall detection by employing the Dynamic Vision Sensor (DVS), which captures intensity changes without compromising individuals’ privacy. This paper introduces a novel event-based dataset named “DVSFall”, incorporating diverse daily living activities (ADL) and simulated falls. Captured from multiple viewpoints using DVS cameras, the dataset encompasses twenty-one participants across varying age groups. To evaluate the dataset, we employed Spiking Neural Networks (SNN) designed to replicate neural activity. Furthermore, we explored a hybrid framework, the 3D-CNN & SNN (NeuCube) approach, for fall detection. Our proposed framework achieved an accuracy of 94.59% with SNN and notably improved to 97.84% using the hybrid approach, as measured against the recorded dataset.

Today, due to the widespread outbreak of the deadly coronavirus, popularly known as COVID-19, the traditional classroom education has been shifted to computer-based learning. Students of various cognitive and psychological abilities participate in the learning process. However, most students are hesitant to provide regular and honest feedback on the comprehensiveness of the course, making it difficult for the instructor to ensure that all students are grasping the information at the same rate. The students’ understanding of the course and their emotional engagement, as indicated via facial expressions, are intertwined. This paper attempts to present a three-dimensional DenseNet self-attention neural network (DenseAttNet) used to identify and evaluate student participation in modern and traditional educational programs. With the Dataset for Affective States in E-Environments (DAiSEE), the proposed DenseAttNet model outperformed all other existing methods, achieving baseline accuracy of 63.59% for engagement classification and 54.27% for boredom classification, respectively. Besides, DenseAttNet trained on all four multi-labels, namely boredom, engagement, confusion, and frustration has registered an accuracy of 81.17%, 94.85%, 90.96%, and 95.85%, respectively. In addition, we performed a regression experiment on DAiSEE and obtained the lowest Mean Square Error (MSE) value of 0.0347. Finally, the proposed approach achieves a competitive MSE of 0.0877 when validated on the Emotion Recognition in the Wild Engagement Prediction (EmotiW-EP) dataset.