ANAnup Nandy

Anup Nandy

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Anup Nandy has a total of 12 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Edge-Based Robot-Assisted Hand Gesture Recognition for Autism Intervention Using Hybrid Keyframe Selection and Contrastive Learning

    by Sougatamoy Biswas, Soumyajeet Mahapatra, Anup Nandy, Asim Kumar Naskar

    Children with Autism Spectrum Disorder (ASD) often face challenges in expressing and interpreting social gestures essential for social skill development. Traditional approaches depend on intrusive wearable sensors and manual supervision, with minimal adaptation to real-time, resource-constrained environments. To address these limitations, we present a real-time vision-based human–robot interaction (HRI) framework that autonomously supports gesture learning and correction without therapist intervention. The system employs MediaPipe-based skeletal landmark extraction with a novel hybrid keyframe selection strategy. This strategy integrates entropy-based filtering to retain semantically rich frames and a clustering-based mechanism to identify representative frames from similar gesture groups. Spatial features are modeled using one-dimensional convolutions across hand joints to capture intra-frame structural dependencies. The framework executes gesture classification and correctness evaluation simultaneously, with contrastive learning incorporated to enhance the separability of learned embeddings. An in-house assistive robot, named CARAT (Caregiver Autism Robot with AI-based Technology), has been developed to realize this framework. CARAT is equipped with a gesture-performing arm, deployed on a Jetson Orin Nano edge device, and provides real-time verbal feedback alongside physical gesture demonstrations. The dataset for evaluation was collected using CARAT, involving participants from both healthy children and children with ASD. The framework demonstrates strong recognition performance, achieving 98.85% accuracy with healthy participants, 96.42% with autistic participants, and 97.41% on the combined dataset of both groups. These results validate the effectiveness of the proposed approach for hand gesture recognition in edge-deployed assistive robots for autism intervention.

  • Embedded AI-Driven Multimodal Human Activity Recognition Using RGB–D Sensor Data and Cross-Modality Attention

    by Sougatamoy Biswas, Anupriya Biswas, Anup Nandy, Asim Kumar Naskar

    Human Activity Recognition (HAR) plays an important role in developing intelligent systems for applications such as assisted living, surveillance, and human-robot interaction. Traditional HAR approaches often rely on a single data modality, such as 2D RGB images or 3D point clouds. While 2D images provide detailed visual and motion cues, they lack depth information. In contrast, 3D data offer rich spatial and depth information but are often low in resolution and lack detailed texture information. These limitations make it challenging to achieve robust activity recognition using a single data modality. To address these challenges, we propose a sensor-driven multimodal HAR framework that integrates RGB and depth data from both 2D and 3D image streams. The system employs dual neural branches for modality-specific feature extraction and introduces a novel Cross-Modality Scaled Dot-Product Attention (CMSDPA) module to adaptively fuse visual and spatial features from 2D and 3D images. The Intel RealSense Multimodal Activity Dataset (IRMAD) used in this study is acquired using the Intel RealSense Depth Camera D435 sensor, which captures synchronized RGB and depth data. The framework is deployed on the Jetson Orin Nano edge AI device and achieves a recognition accuracy of 98.15%. Experimental results demonstrate that our model effectively leverages multimodal sensor data, enabling accurate, robust, and deployable HAR in real-world environments.

  • Lightweight Multimodal Feature Fusion and Spatiotemporal Learning for Human Action Recognition on Edge Devices

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar

    Human action recognition (HAR) remains a challenging topic in computer vision, attracting extensive research for applications in surveillance, sports, and human-computer interaction. Existing deep learning-based HAR methods often rely on either RGB-only inputs or global attention mechanisms, which suffer from poor generalization under occlusion, background clutter, and temporal ambiguity. Moreover, conventional methods struggle to capture fine-grained spatiotemporal dependencies due to sudden changes in motion dynamics and the lack of temporal consistency constraints in sequential modeling. To address these limitations, we propose a lightweight multimodal human action recognition framework that combines complementary cues from appearance, motion, and depth modalities. These features are integrated through a mid-level feature fusion strategy to form a unified and discriminative representation of human actions. The architecture employs an Enhanced Long-Term Recurrent Convolutional Network (E-LRCN) to model both spatial and temporal dynamics efficiently. A novel Temporal Causal Self-Attention (TCSA) module is introduced to enforce directional temporal consistency. It emphasizes recent motion context, significantly improving the discrimination of action sequences. Extensive evaluations on the KTH, UCF-101, JHMDB, and HMDB51 datasets show that the proposed framework surpasses state-of-the-art methods, achieving accuracies of 98.10%, 96.28%, 83.47%, and 77.60%, respectively. These results reflect gains of up to 1.27%, 2.08%, 2.81%, and 1.04% over the best-performing benchmark models. The proposed framework improves performance while reducing computational overhead, making it suitable for real-world human action recognition on resource-constrained platforms like Jetson Nano.


  • MediaPipe with LSTM Architecture for Real-Time Hand Gesture Recognization

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar, Rahul Saw


  • Semi-supervised Video Object Segmentation Using Parallel Coattention Network

    by Sangramjit Chakraborty, Monalisha Mahapatra, Anup Nandy

  • Object Detection with YOLO Model on NAO Humanoid Robot

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar

  • Open Access

    Gait Event Prediction of People with Cerebral Palsy using Feature Uncertainty: A Low-Cost Approach

    by Saikat Chakraborty, Noble Thomas, Anup Nandy

    Incorporation of feature uncertainty during model construction explores the real generalization ability of that model. But this factor has been avoided often during automatic gait event detection for Cerebral Palsy patients. Again, the prevailing vision-based gait event detection systems are expensive due to incorporation of high-end motion tracking cameras. This study proposes a low-cost gait event detection system for heel strike and toe-off events. A state-space model was constructed where the temporal evolution of gait signal was devised by quantifying feature uncertainty. The model was trained using Cardiff classifier. Ankle velocity was taken as the input feature. The frame associated with state transition was marked as a gait event. The model was tested on 15 Cerebral Palsy patients and 15 normal subjects. Data acquisition was performed using low-cost Kinect cameras. The model identified gait events on an average of 2 frame error. All events were predicted before the actual occurrence. Error for toe-off was less than the heel strike. Incorporation of the uncertainty factor in the detection of gait events exhibited a competing performance with respect to state-of-the-art.