H. Pallab Jyoti Dutta
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
H. Pallab Jyoti Dutta has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Manas Kamal Bhuyan
Indian Institute of Technology Guwahati
6 shared publications - Rabul Hussain Laskar
National Institute of Technology Silchar
4 shared publications - Debanga Raj Neog
Mehta Family School of Data Science and Artificial Intelligence, Indian Institute of Technology Guwahati, Guwahati, India
3 shared publications - Karl Fredric MacDorman
Indiana University
2 shared publications M. K. Bhuyan
IIT Guwahati
2 shared publicationsBishshoy Das
IIT Guwahati
1 shared publication
Latest Publications
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 8 (9) , 1-4
Attention-Based 2-D Hand Keypoints Localization
Hand keypoint localization is used extensively in human–computer interaction, but accurate localization is challenging due to closeness between the fingers and the keypoints, occlusion, varied hand poses, complex backgrounds, and extreme lighting conditions. Despite much research, challenges persist. Therefore, we propose an encoder–decoder architecture aided by a novel attention module to precisely localize hand keypoints. The attention module captures keypoint-relevant features at two different scales that encompass local and global characteristics. Further, the loss function teaches the model to remove spurious detected keypoints in the initial learning phase. The proposed architecture outputs precise keypoint locations, as indicated by the qualitative and quantitative results. Evaluation of two benchmark RGB image datasets, comprising all the challenges encountered in keypoint localization, resulted in endpoint errors as low as 2.78 and 1.85 pixels and 98.50% and 99.77% correct keypoints, respectively. This shows the proposed model's effectiveness and ability to overcome challenges.
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 8 (9) , 1-4
Attention-Based 2-D Hand Keypoints Localization
Hand keypoint localization is used extensively in human–computer interaction, but accurate localization is challenging due to closeness between the fingers and the keypoints, occlusion, varied hand poses, complex backgrounds, and extreme lighting conditions. Despite much research, challenges persist. Therefore, we propose an encoder–decoder architecture aided by a novel attention module to precisely localize hand keypoints. The attention module captures keypoint-relevant features at two different scales that encompass local and global characteristics. Further, the loss function teaches the model to remove spurious detected keypoints in the initial learning phase. The proposed architecture outputs precise keypoint locations, as indicated by the qualitative and quantitative results. Evaluation of two benchmark RGB image datasets, comprising all the challenges encountered in keypoint localization, resulted in endpoint errors as low as 2.78 and 1.85 pixels and 98.50% and 99.77% correct keypoints, respectively. This shows the proposed model's effectiveness and ability to overcome challenges.
- 12 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-11
Multiscale Attention-Based Hand Keypoint Detection
This paper deals with the measurement of the hand keypoints in a vision-based setup under different constraints. Hand keypoint detection plays a crucial role in many gesture-based applications. However, developing a generalized detection method has remained a long-standing problem. Several factors impede accurate detection: the fingers’ distance from the camera and their nearness, self-occlusion, variations in illumination, and background clutter. To overcome these barriers, we propose a two-stage architecture. The first stage generates precise hand regions, eliminating adjoining skin regions and background clutter. The second stage incorporates a novel multiscale attention block to detect keypoint coordinates precisely. Qualitative and quantitative evaluations found that the proposed architecture outperforms state-of-the-art models, with endpoint errors as low as 2.3, 1.14, and 2.11 pixels for the three benchmark datasets. This advancement lays the groundwork for future 3D hand pose estimation developments and their applications.
- 12 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 73 (00189456) , 1-11
Multiscale Attention-Based Hand Keypoint Detection
This paper deals with the measurement of the hand keypoints in a vision-based setup under different constraints. Hand keypoint detection plays a crucial role in many gesture-based applications. However, developing a generalized detection method has remained a long-standing problem. Several factors impede accurate detection: the fingers’ distance from the camera and their nearness, self-occlusion, variations in illumination, and background clutter. To overcome these barriers, we propose a two-stage architecture. The first stage generates precise hand regions, eliminating adjoining skin regions and background clutter. The second stage incorporates a novel multiscale attention block to detect keypoint coordinates precisely. Qualitative and quantitative evaluations found that the proposed architecture outperforms state-of-the-art models, with endpoint errors as low as 2.3, 1.14, and 2.11 pixels for the three benchmark datasets. This advancement lays the groundwork for future 3D hand pose estimation developments and their applications.
- 28 July 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 234
Efficient hand segmentation for rehabilitation tasks using a convolution neural network with attention
- 28 July 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 234
Efficient hand segmentation for rehabilitation tasks using a convolution neural network with attention
- 3 July 2023
- Published by Springer Nature in Evolving Systems
- Vol. 15 (1) , 185-201
Attention-based hand semantic segmentation and gesture recognition using deep networks
- 3 July 2023
- Published by Springer Nature in Evolving Systems
- Vol. 15 (1) , 185-201
Attention-based hand semantic segmentation and gesture recognition using deep networks
- 5 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-13
Patient Assistance System Based on Hand Gesture Recognition
We propose a two-stage hand gesture recognition architecture to support a patient assistance system. Some medical conditions limit mobility, and the patient must rely on medical staff to meet their needs. In such cases, a phone or intercom is not convenient to call for help. A vision-based system operated by changing the orientation of fingers can be used to send specific messages without making arm movements. However, vision-based hand gesture recognition is hindered by occlusion, background clutter, and variations in illumination. Therefore, we developed a two-stage architecture: The first stage produces a saliency map to simplify recognition and the second stage performs classification. A novel combined loss function optimizes the saliency detection model and makes the saliency map more precise. An adaptive kernel-based channel attention layer is used to emphasize salient features. The proposed architecture achieved precise saliency detection on four benchmark datasets and high-accuracy recognition on two. We designed an interface for patients to send specific messages to the medical staff using hand gestures. The interface help patients request assistance and connect with medical staff without leaving the bed or involving a third party.
- 5 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 72 (00189456) , 1-13
Patient Assistance System Based on Hand Gesture Recognition
We propose a two-stage hand gesture recognition architecture to support a patient assistance system. Some medical conditions limit mobility, and the patient must rely on medical staff to meet their needs. In such cases, a phone or intercom is not convenient to call for help. A vision-based system operated by changing the orientation of fingers can be used to send specific messages without making arm movements. However, vision-based hand gesture recognition is hindered by occlusion, background clutter, and variations in illumination. Therefore, we developed a two-stage architecture: The first stage produces a saliency map to simplify recognition and the second stage performs classification. A novel combined loss function optimizes the saliency detection model and makes the saliency map more precise. An adaptive kernel-based channel attention layer is used to emphasize salient features. The proposed architecture achieved precise saliency detection on four benchmark datasets and high-accuracy recognition on two. We designed an interface for patients to send specific messages to the medical staff using hand gestures. The interface help patients request assistance and connect with medical staff without leaving the bed or involving a third party.
- 2 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 5 (2) , 698-708
A Hand Gesture-Operated System for Rehabilitation Using an End-to-End Detection Framework
We propose a hand gesture-operated system as an AI application to relieve discomfort and restore function in hand and arm movements caused by injuries and nerve and muscle complications. The system trains patients with hand exercises, such as performing hand gestures accurately, traversing within specified bounds, and operating a hand gesture calculator. However, the system requires accurate hand gesture detection, which is impeded by background clutter and variations in illumination and in the hand's size and angle. To address this, we developed a robust hand detection module that uses a single-stage transformer deep network. The transformer network encodes global information and uses bipartite matching to reduce the frequency of spurious detections. It drives a regression head and a classification head to localize the hand gesture in a bounding box and assign it a class label. Hand keypoints are also detected to support drawing, path traversal, and calculator use. The approach is evaluated on two benchmark datasets: OUHANDS and NUS. The method yields 89.6% accuracy for OUHANDS and 100% for NUS. These results indicate that precise hand detection can support a robust system for rehabilitation through hand exercises. Our experiments confirm that the users' hand function progressively improved.
- 2 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Artificial Intelligence
- Vol. 5 (2) , 698-708
A Hand Gesture-Operated System for Rehabilitation Using an End-to-End Detection Framework
We propose a hand gesture-operated system as an AI application to relieve discomfort and restore function in hand and arm movements caused by injuries and nerve and muscle complications. The system trains patients with hand exercises, such as performing hand gestures accurately, traversing within specified bounds, and operating a hand gesture calculator. However, the system requires accurate hand gesture detection, which is impeded by background clutter and variations in illumination and in the hand's size and angle. To address this, we developed a robust hand detection module that uses a single-stage transformer deep network. The transformer network encodes global information and uses bipartite matching to reduce the frequency of spurious detections. It drives a regression head and a classification head to localize the hand gesture in a bounding box and assign it a class label. Hand keypoints are also detected to support drawing, path traversal, and calculator use. The approach is evaluated on two benchmark datasets: OUHANDS and NUS. The method yields 89.6% accuracy for OUHANDS and 100% for NUS. These results indicate that precise hand detection can support a robust system for rehabilitation through hand exercises. Our experiments confirm that the users' hand function progressively improved.
- 24 July 2022
- Published by Springer Nature
- p. 438-449
Hand Gesture Recognition Using CBAM-RetinaNet
- 26 March 2021
- Published by Springer Nature
- p. 419-433
Graph-Based Depth Estimation in a Monocular Image Using Constrained 3D Wireframe Models
- 26 March 2021
- Published by Springer Nature
- p. 419-433
Graph-Based Depth Estimation in a Monocular Image Using Constrained 3D Wireframe Models
- 26 March 2021
- Published by Springer Nature
- p. 291-304
An Unsupervised Approach for Estimating Depth of Outdoor Scenes from Monocular Image
- 26 March 2021
- Published by Springer Nature
- p. 291-304
An Unsupervised Approach for Estimating Depth of Outdoor Scenes from Monocular Image