SBSougatamoy Biswas

Sougatamoy Biswas

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Sougatamoy Biswas has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Edge-Based Robot-Assisted Hand Gesture Recognition for Autism Intervention Using Hybrid Keyframe Selection and Contrastive Learning

    by Sougatamoy Biswas, Soumyajeet Mahapatra, Anup Nandy, Asim Kumar Naskar

    Children with Autism Spectrum Disorder (ASD) often face challenges in expressing and interpreting social gestures essential for social skill development. Traditional approaches depend on intrusive wearable sensors and manual supervision, with minimal adaptation to real-time, resource-constrained environments. To address these limitations, we present a real-time vision-based human–robot interaction (HRI) framework that autonomously supports gesture learning and correction without therapist intervention. The system employs MediaPipe-based skeletal landmark extraction with a novel hybrid keyframe selection strategy. This strategy integrates entropy-based filtering to retain semantically rich frames and a clustering-based mechanism to identify representative frames from similar gesture groups. Spatial features are modeled using one-dimensional convolutions across hand joints to capture intra-frame structural dependencies. The framework executes gesture classification and correctness evaluation simultaneously, with contrastive learning incorporated to enhance the separability of learned embeddings. An in-house assistive robot, named CARAT (Caregiver Autism Robot with AI-based Technology), has been developed to realize this framework. CARAT is equipped with a gesture-performing arm, deployed on a Jetson Orin Nano edge device, and provides real-time verbal feedback alongside physical gesture demonstrations. The dataset for evaluation was collected using CARAT, involving participants from both healthy children and children with ASD. The framework demonstrates strong recognition performance, achieving 98.85% accuracy with healthy participants, 96.42% with autistic participants, and 97.41% on the combined dataset of both groups. These results validate the effectiveness of the proposed approach for hand gesture recognition in edge-deployed assistive robots for autism intervention.

  • Embedded AI-Driven Multimodal Human Activity Recognition Using RGB–D Sensor Data and Cross-Modality Attention

    by Sougatamoy Biswas, Anupriya Biswas, Anup Nandy, Asim Kumar Naskar

    Human Activity Recognition (HAR) plays an important role in developing intelligent systems for applications such as assisted living, surveillance, and human-robot interaction. Traditional HAR approaches often rely on a single data modality, such as 2D RGB images or 3D point clouds. While 2D images provide detailed visual and motion cues, they lack depth information. In contrast, 3D data offer rich spatial and depth information but are often low in resolution and lack detailed texture information. These limitations make it challenging to achieve robust activity recognition using a single data modality. To address these challenges, we propose a sensor-driven multimodal HAR framework that integrates RGB and depth data from both 2D and 3D image streams. The system employs dual neural branches for modality-specific feature extraction and introduces a novel Cross-Modality Scaled Dot-Product Attention (CMSDPA) module to adaptively fuse visual and spatial features from 2D and 3D images. The Intel RealSense Multimodal Activity Dataset (IRMAD) used in this study is acquired using the Intel RealSense Depth Camera D435 sensor, which captures synchronized RGB and depth data. The framework is deployed on the Jetson Orin Nano edge AI device and achieves a recognition accuracy of 98.15%. Experimental results demonstrate that our model effectively leverages multimodal sensor data, enabling accurate, robust, and deployable HAR in real-world environments.

  • Lightweight Multimodal Feature Fusion and Spatiotemporal Learning for Human Action Recognition on Edge Devices

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar

    Human action recognition (HAR) remains a challenging topic in computer vision, attracting extensive research for applications in surveillance, sports, and human-computer interaction. Existing deep learning-based HAR methods often rely on either RGB-only inputs or global attention mechanisms, which suffer from poor generalization under occlusion, background clutter, and temporal ambiguity. Moreover, conventional methods struggle to capture fine-grained spatiotemporal dependencies due to sudden changes in motion dynamics and the lack of temporal consistency constraints in sequential modeling. To address these limitations, we propose a lightweight multimodal human action recognition framework that combines complementary cues from appearance, motion, and depth modalities. These features are integrated through a mid-level feature fusion strategy to form a unified and discriminative representation of human actions. The architecture employs an Enhanced Long-Term Recurrent Convolutional Network (E-LRCN) to model both spatial and temporal dynamics efficiently. A novel Temporal Causal Self-Attention (TCSA) module is introduced to enforce directional temporal consistency. It emphasizes recent motion context, significantly improving the discrimination of action sequences. Extensive evaluations on the KTH, UCF-101, JHMDB, and HMDB51 datasets show that the proposed framework surpasses state-of-the-art methods, achieving accuracies of 98.10%, 96.28%, 83.47%, and 77.60%, respectively. These results reflect gains of up to 1.27%, 2.08%, 2.81%, and 1.04% over the best-performing benchmark models. The proposed framework improves performance while reducing computational overhead, making it suitable for real-world human action recognition on resource-constrained platforms like Jetson Nano.


  • MediaPipe with LSTM Architecture for Real-Time Hand Gesture Recognization

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar, Rahul Saw


  • Object Detection with YOLO Model on NAO Humanoid Robot

    by Sougatamoy Biswas, Anup Nandy, Asim Kumar Naskar

  • Graph Neural Network and Its Applications

    by Sougatamoy Biswas

    Graph neural network (GNN) is an emerging field in deep learning. Graphs have more expressive power than any other data structure. Graph neural network is one of the application areas of deep learning, and it has applications in different domains where traditional convolutional neural networks can't give the desired result. Graphs are basically connections of nodes through the edges. In the area of recommendation systems, image processing and fraud detection are some of the few application areas of graph neural networks. As graphs are moveable and mobile in nature, they are more flexible to apply in these domains. GNN deals with these types of problems more effectively than a convolution neural network. To apply GNN to a specific problem domain, data needs to be converted into a graphical format, and then neural network operations can be executed. The main feature of GNN is to inherit information from its neighborhood. This is called graph embedding. This chapter describes basic GNN architecture, GNN advantage over CNN, and its application in different domains.

  • Open Access

    PCA-Based Incremental Extreme Learning Machine (PCA-IELM) for COVID-19 Patient Diagnosis Using Chest X-Ray Images

    by Vinod Kumar, Sougatamoy Biswas, Dharmendra Singh Rajput, Harshita Patel, Basant Tiwari

    Novel coronavirus 2019 has created a pandemic and was first reported in December 2019. It has had very adverse consequences on people's daily life, healthcare, and the world's economy as well. According to the World Health Organization's most recent statistics, COVID-19 has become a worldwide pandemic, and the number of infected persons and fatalities growing at an alarming rate. It is highly required to have an effective system to early detect the COVID-19 patients to curb the further spreading of the virus from the affected person. Therefore, to early identify positive cases in patients and to support radiologists in the automatic diagnosis of COVID-19 from X-ray images, a novel method PCA-IELM is proposed based on principal component analysis (PCA) and incremental extreme learning machine. The suggested method's key addition is that it considers the benefits of PCA and the incremental extreme learning machine. Further, our strategy PCA-IELM reduces the input dimension by extracting the most important information from an image. Consequently, the technique can effectively increase the COVID-19 patient prediction performance. In addition to these, PCA-IELM has a faster training speed than a multi-layer neural network. The proposed approach was tested on a COVID-19 patient’s chest X-ray image dataset. The experimental results indicate that the proposed approach PCA-IELM outperforms PCA-SVM and PCA-ELM in terms of accuracy (98.11%), precision (96.11%), recall (97.50%), F1-score (98.50%), etc., and training speed.