Anup Nandy
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Anup Nandy has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Asim Kumar Naskar
Department of Electrical Engineering, National Institute of Technology Rourkela, Rourkela, India
5 shared publications - Sougatamoy Biswas
Department of Computer Science and Engineering, National Institute of Technology Rourkela, Rourkela, India
5 shared publications Anupriya Biswas
Department of Computer Science and Engineering, National Institute of Technology Rourkela, Odisha, India
1 shared publication- Arupananda Sahoo
National Institute of Technology
1 shared publication - Mainak Ghosh
Department of Computer Science and Engineering, National Institute of Technology Rourkela, Rourkela, India
1 shared publication - Monalisha Mahapatra
National Institute of Technology, Rourkela
1 shared publication
Latest Publications
- 17 April 2026
- Published by Springer Nature
- p. 172-180
Hybrid Neural Model for Classification of Neurodegenerative Gait Disorders
- 17 April 2026
- Published by Springer Nature
- p. 172-180
Hybrid Neural Model for Classification of Neurodegenerative Gait Disorders
- 13 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 8 (2) , 811-822
Edge-Based Robot-Assisted Hand Gesture Recognition for Autism Intervention Using Hybrid Keyframe Selection and Contrastive Learning
Children with Autism Spectrum Disorder (ASD) often face challenges in expressing and interpreting social gestures essential for social skill development. Traditional approaches depend on intrusive wearable sensors and manual supervision, with minimal adaptation to real-time, resource-constrained environments. To address these limitations, we present a real-time vision-based human–robot interaction (HRI) framework that autonomously supports gesture learning and correction without therapist intervention. The system employs MediaPipe-based skeletal landmark extraction with a novel hybrid keyframe selection strategy. This strategy integrates entropy-based filtering to retain semantically rich frames and a clustering-based mechanism to identify representative frames from similar gesture groups. Spatial features are modeled using one-dimensional convolutions across hand joints to capture intra-frame structural dependencies. The framework executes gesture classification and correctness evaluation simultaneously, with contrastive learning incorporated to enhance the separability of learned embeddings. An in-house assistive robot, named CARAT (Caregiver Autism Robot with AI-based Technology), has been developed to realize this framework. CARAT is equipped with a gesture-performing arm, deployed on a Jetson Orin Nano edge device, and provides real-time verbal feedback alongside physical gesture demonstrations. The dataset for evaluation was collected using CARAT, involving participants from both healthy children and children with ASD. The framework demonstrates strong recognition performance, achieving 98.85% accuracy with healthy participants, 96.42% with autistic participants, and 97.41% on the combined dataset of both groups. These results validate the effectiveness of the proposed approach for hand gesture recognition in edge-deployed assistive robots for autism intervention.
- 13 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Robotics and Bionics
- Vol. 8 (2) , 811-822
Edge-Based Robot-Assisted Hand Gesture Recognition for Autism Intervention Using Hybrid Keyframe Selection and Contrastive Learning
Children with Autism Spectrum Disorder (ASD) often face challenges in expressing and interpreting social gestures essential for social skill development. Traditional approaches depend on intrusive wearable sensors and manual supervision, with minimal adaptation to real-time, resource-constrained environments. To address these limitations, we present a real-time vision-based human–robot interaction (HRI) framework that autonomously supports gesture learning and correction without therapist intervention. The system employs MediaPipe-based skeletal landmark extraction with a novel hybrid keyframe selection strategy. This strategy integrates entropy-based filtering to retain semantically rich frames and a clustering-based mechanism to identify representative frames from similar gesture groups. Spatial features are modeled using one-dimensional convolutions across hand joints to capture intra-frame structural dependencies. The framework executes gesture classification and correctness evaluation simultaneously, with contrastive learning incorporated to enhance the separability of learned embeddings. An in-house assistive robot, named CARAT (Caregiver Autism Robot with AI-based Technology), has been developed to realize this framework. CARAT is equipped with a gesture-performing arm, deployed on a Jetson Orin Nano edge device, and provides real-time verbal feedback alongside physical gesture demonstrations. The dataset for evaluation was collected using CARAT, involving participants from both healthy children and children with ASD. The framework demonstrates strong recognition performance, achieving 98.85% accuracy with healthy participants, 96.42% with autistic participants, and 97.41% on the combined dataset of both groups. These results validate the effectiveness of the proposed approach for hand gesture recognition in edge-deployed assistive robots for autism intervention.
- 18 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. PP (1530437X) , 1
Embedded AI-Driven Multimodal Human Activity Recognition Using RGB–D Sensor Data and Cross-Modality Attention
Human Activity Recognition (HAR) plays an important role in developing intelligent systems for applications such as assisted living, surveillance, and human-robot interaction. Traditional HAR approaches often rely on a single data modality, such as 2D RGB images or 3D point clouds. While 2D images provide detailed visual and motion cues, they lack depth information. In contrast, 3D data offer rich spatial and depth information but are often low in resolution and lack detailed texture information. These limitations make it challenging to achieve robust activity recognition using a single data modality. To address these challenges, we propose a sensor-driven multimodal HAR framework that integrates RGB and depth data from both 2D and 3D image streams. The system employs dual neural branches for modality-specific feature extraction and introduces a novel Cross-Modality Scaled Dot-Product Attention (CMSDPA) module to adaptively fuse visual and spatial features from 2D and 3D images. The Intel RealSense Multimodal Activity Dataset (IRMAD) used in this study is acquired using the Intel RealSense Depth Camera D435 sensor, which captures synchronized RGB and depth data. The framework is deployed on the Jetson Orin Nano edge AI device and achieves a recognition accuracy of 98.15%. Experimental results demonstrate that our model effectively leverages multimodal sensor data, enabling accurate, robust, and deployable HAR in real-world environments.
- 18 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. PP (1530437X) , 1
Embedded AI-Driven Multimodal Human Activity Recognition Using RGB–D Sensor Data and Cross-Modality Attention
Human Activity Recognition (HAR) plays an important role in developing intelligent systems for applications such as assisted living, surveillance, and human-robot interaction. Traditional HAR approaches often rely on a single data modality, such as 2D RGB images or 3D point clouds. While 2D images provide detailed visual and motion cues, they lack depth information. In contrast, 3D data offer rich spatial and depth information but are often low in resolution and lack detailed texture information. These limitations make it challenging to achieve robust activity recognition using a single data modality. To address these challenges, we propose a sensor-driven multimodal HAR framework that integrates RGB and depth data from both 2D and 3D image streams. The system employs dual neural branches for modality-specific feature extraction and introduces a novel Cross-Modality Scaled Dot-Product Attention (CMSDPA) module to adaptively fuse visual and spatial features from 2D and 3D images. The Intel RealSense Multimodal Activity Dataset (IRMAD) used in this study is acquired using the Intel RealSense Depth Camera D435 sensor, which captures synchronized RGB and depth data. The framework is deployed on the Jetson Orin Nano edge AI device and achieves a recognition accuracy of 98.15%. Experimental results demonstrate that our model effectively leverages multimodal sensor data, enabling accurate, robust, and deployable HAR in real-world environments.
- 14 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 10 (1) , 982-995
Lightweight Multimodal Feature Fusion and Spatiotemporal Learning for Human Action Recognition on Edge Devices
Human action recognition (HAR) remains a challenging topic in computer vision, attracting extensive research for applications in surveillance, sports, and human-computer interaction. Existing deep learning-based HAR methods often rely on either RGB-only inputs or global attention mechanisms, which suffer from poor generalization under occlusion, background clutter, and temporal ambiguity. Moreover, conventional methods struggle to capture fine-grained spatiotemporal dependencies due to sudden changes in motion dynamics and the lack of temporal consistency constraints in sequential modeling. To address these limitations, we propose a lightweight multimodal human action recognition framework that combines complementary cues from appearance, motion, and depth modalities. These features are integrated through a mid-level feature fusion strategy to form a unified and discriminative representation of human actions. The architecture employs an Enhanced Long-Term Recurrent Convolutional Network (E-LRCN) to model both spatial and temporal dynamics efficiently. A novel Temporal Causal Self-Attention (TCSA) module is introduced to enforce directional temporal consistency. It emphasizes recent motion context, significantly improving the discrimination of action sequences. Extensive evaluations on the KTH, UCF-101, JHMDB, and HMDB51 datasets show that the proposed framework surpasses state-of-the-art methods, achieving accuracies of 98.10%, 96.28%, 83.47%, and 77.60%, respectively. These results reflect gains of up to 1.27%, 2.08%, 2.81%, and 1.04% over the best-performing benchmark models. The proposed framework improves performance while reducing computational overhead, making it suitable for real-world human action recognition on resource-constrained platforms like Jetson Nano.
- 14 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 10 (1) , 982-995
Lightweight Multimodal Feature Fusion and Spatiotemporal Learning for Human Action Recognition on Edge Devices
Human action recognition (HAR) remains a challenging topic in computer vision, attracting extensive research for applications in surveillance, sports, and human-computer interaction. Existing deep learning-based HAR methods often rely on either RGB-only inputs or global attention mechanisms, which suffer from poor generalization under occlusion, background clutter, and temporal ambiguity. Moreover, conventional methods struggle to capture fine-grained spatiotemporal dependencies due to sudden changes in motion dynamics and the lack of temporal consistency constraints in sequential modeling. To address these limitations, we propose a lightweight multimodal human action recognition framework that combines complementary cues from appearance, motion, and depth modalities. These features are integrated through a mid-level feature fusion strategy to form a unified and discriminative representation of human actions. The architecture employs an Enhanced Long-Term Recurrent Convolutional Network (E-LRCN) to model both spatial and temporal dynamics efficiently. A novel Temporal Causal Self-Attention (TCSA) module is introduced to enforce directional temporal consistency. It emphasizes recent motion context, significantly improving the discrimination of action sequences. Extensive evaluations on the KTH, UCF-101, JHMDB, and HMDB51 datasets show that the proposed framework surpasses state-of-the-art methods, achieving accuracies of 98.10%, 96.28%, 83.47%, and 77.60%, respectively. These results reflect gains of up to 1.27%, 2.08%, 2.81%, and 1.04% over the best-performing benchmark models. The proposed framework improves performance while reducing computational overhead, making it suitable for real-world human action recognition on resource-constrained platforms like Jetson Nano.
- 3 July 2024
- Published by Springer Nature
- p. 422-431
MediaPipe with LSTM Architecture for Real-Time Hand Gesture Recognization
- 3 July 2024
- Published by Springer Nature
- p. 422-431
MediaPipe with LSTM Architecture for Real-Time Hand Gesture Recognization
- 4 December 2023
- Published by Springer Nature
- p. 449-456
Semi-supervised Video Object Segmentation Using Parallel Coattention Network
- 4 December 2023
- Published by Springer Nature
- p. 449-456
Semi-supervised Video Object Segmentation Using Parallel Coattention Network
- 4 December 2023
- Published by Springer Nature
- p. 495-502
Object Detection with YOLO Model on NAO Humanoid Robot
- 4 December 2023
- Published by Springer Nature
- p. 495-502
Object Detection with YOLO Model on NAO Humanoid Robot
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 301-306
Gait Event Prediction of People with Cerebral Palsy using Feature Uncertainty: A Low-Cost Approach
Incorporation of feature uncertainty during model construction explores the real generalization ability of that model. But this factor has been avoided often during automatic gait event detection for Cerebral Palsy patients. Again, the prevailing vision-based gait event detection systems are expensive due to incorporation of high-end motion tracking cameras. This study proposes a low-cost gait event detection system for heel strike and toe-off events. A state-space model was constructed where the temporal evolution of gait signal was devised by quantifying feature uncertainty. The model was trained using Cardiff classifier. Ankle velocity was taken as the input feature. The frame associated with state transition was marked as a gait event. The model was tested on 15 Cerebral Palsy patients and 15 normal subjects. Data acquisition was performed using low-cost Kinect cameras. The model identified gait events on an average of 2 frame error. All events were predicted before the actual occurrence. Error for toe-off was less than the heel strike. Incorporation of the uncertainty factor in the detection of gait events exhibited a competing performance with respect to state-of-the-art.
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 301-306
Open AccessGait Event Prediction of People with Cerebral Palsy using Feature Uncertainty: A Low-Cost Approach
Incorporation of feature uncertainty during model construction explores the real generalization ability of that model. But this factor has been avoided often during automatic gait event detection for Cerebral Palsy patients. Again, the prevailing vision-based gait event detection systems are expensive due to incorporation of high-end motion tracking cameras. This study proposes a low-cost gait event detection system for heel strike and toe-off events. A state-space model was constructed where the temporal evolution of gait signal was devised by quantifying feature uncertainty. The model was trained using Cardiff classifier. Ankle velocity was taken as the input feature. The frame associated with state transition was marked as a gait event. The model was tested on 15 Cerebral Palsy patients and 15 normal subjects. Data acquisition was performed using low-cost Kinect cameras. The model identified gait events on an average of 2 frame error. All events were predicted before the actual occurrence. Error for toe-off was less than the heel strike. Incorporation of the uncertainty factor in the detection of gait events exhibited a competing performance with respect to state-of-the-art.
- 28 March 2021
- Published by Springer Nature
- p. 330-340
Human Gait Abnormality Detection Using Low Cost Sensor Technology
- 28 March 2021
- Published by Springer Nature
- p. 330-340
Human Gait Abnormality Detection Using Low Cost Sensor Technology