Debashis Ghosh
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Debashis Ghosh has a total of 87 co-authors. Here are the top co-authors by number of shared publications:
H. Roy
CSIR-Central Mechanical Engineering Research Institute
11 shared publications- Pyari Mohan Pradhan
Department of Electronics and Communication Engineering, Indian Institute of Technology Roorkee, Roorkee, India
11 shared publications S. K. Mitra
Metallurgical and Materials Engineering Department, NIT Durgapur, Durgapur, India
10 shared publications- Mallika Garg
Department of Electronics and Communication Engineering Indian Institute of Technology Roorkee India
9 shared publications A. K. Shukla
7 shared publications- Priya Gupta
Department of Electronics and Communication Engineering, Indian Institute of Technology (IIT) Roorkee, Roorkee, India
6 shared publications
Latest Publications
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 10  (1) , 1-4
Multiview Hand Gesture Recognition From Generated Gestures Using Conditional Adversarial Network
Hand gesture recognition systems often struggles when a gesture is occluded by the hand itself or by the other hand. To address this problem, multiview gestures may be used while training a gesture recognition system. Unfortunately, there are very few datasets available that contains different views of every single gesture. In this paper, we propose a method to handle occlusions in single-view sign gestures by generating multiple views from a single-view gesture using conditional multiview gesture synthesis. The generated views help to solve the occlusion problem, thereby enhancing the recognition performance. Additionally, we introduce a vision-based Multiview Hand Gesture Recognition (MHGR) framework that utilizes the generated multiview gestures for gesture recognition. Experiments conducted on the HGM-4 dataset demonstrate that the generated images are of high quality, photo-realistic, and significantly improve the recognition accuracy compared to some other existing methods.
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 10  (1) , 1-4
Multiview Hand Gesture Recognition From Generated Gestures Using Conditional Adversarial Network
Hand gesture recognition systems often struggles when a gesture is occluded by the hand itself or by the other hand. To address this problem, multiview gestures may be used while training a gesture recognition system. Unfortunately, there are very few datasets available that contains different views of every single gesture. In this paper, we propose a method to handle occlusions in single-view sign gestures by generating multiple views from a single-view gesture using conditional multiview gesture synthesis. The generated views help to solve the occlusion problem, thereby enhancing the recognition performance. Additionally, we introduce a vision-based Multiview Hand Gesture Recognition (MHGR) framework that utilizes the generated multiview gestures for gesture recognition. Experiments conducted on the HGM-4 dataset demonstrate that the generated images are of high quality, photo-realistic, and significantly improve the recognition accuracy compared to some other existing methods.
- 12 May 2025
- Published by Springer Nature
- p. 15-33
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
- 12 May 2025
- Published by Springer Nature
- p. 15-33
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
- Open Access
OccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.
Open AccessOccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.
- 13 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 74  (6) , 9967-9972
Full-Duplex Massive MIMO Relaying Over Spatially Correlated Rician Channel With Phase Shifts
We analytically study a multi-pair two-way full-duplex (FD) massive multiple-input multiple-output relay system over a spatially correlated Rician channel with unknown phase shifts. Most of the existing literature ignores the line-of-sight (LoS) phase shift, which arises due to user mobility and hardware impairments. We analyze the impact of LoS phase shift on the spectral efficiency (SE) of the system by considering two different channel estimators: i) phase-unaware linear minimum mean square error (LMMSE) and ii) phase-aware MMSE. We also formulate the asymptotic SE expressions for three different power scaling schemes considering zero-forcing receiver/zero-forcing transmitter processing at the relay. We numerically show the impact of unknown phase shifts on the SE of the system. Finally, we highlight the impact of i) loop interference due to the FD operation and ii) pilot power on the sum-SE.
- 13 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 74  (6) , 9967-9972
Full-Duplex Massive MIMO Relaying Over Spatially Correlated Rician Channel With Phase Shifts
We analytically study a multi-pair two-way full-duplex (FD) massive multiple-input multiple-output relay system over a spatially correlated Rician channel with unknown phase shifts. Most of the existing literature ignores the line-of-sight (LoS) phase shift, which arises due to user mobility and hardware impairments. We analyze the impact of LoS phase shift on the spectral efficiency (SE) of the system by considering two different channel estimators: i) phase-unaware linear minimum mean square error (LMMSE) and ii) phase-aware MMSE. We also formulate the asymptotic SE expressions for three different power scaling schemes considering zero-forcing receiver/zero-forcing transmitter processing at the relay. We numerically show the impact of unknown phase shifts on the SE of the system. Finally, we highlight the impact of i) loop interference due to the FD operation and ii) pilot power on the sum-SE.
- Open Access
Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22\% and 99.10\% on NVGesture and Briareo datasets, respectively.
Open AccessMultiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22\% and 99.10\% on NVGesture and Briareo datasets, respectively.
- Open Access
- 24 November 2024
- Published by Springer Nature in Cureus Journal of Computer Science
- Vol. 16  (11)
Development of an Animation-Based Indian Sign Language e-Learning App for Persons With Hearing and Speech Impairment
Human-computer interaction (HCI) is changing the quality of our lives. In this paper, we present one such application of HCI that comes as a boon in the life of a hearing-impaired person. We propose to build an e-learning tool for the education of the hearing- and speech-impaired community by creating animated gestures in sign language, with particular emphasis on the Indian Sign Language (ISL). With the help of MATLAB App Designer, we developed a standalone MATLAB application for ISL fingerspelling (including digits, alphabets and combination of these two) and some ISL words of daily usage. This app can be used by the hearing-impaired community for learning ISL, even while seated at home, or for training signers and teachers at special schools. In our proposed method, the software first learns different sign gestures from real gesture videos. For this, hand regions in every frame are extracted using image processing algorithms and subsequently identifying key video frames that show significant change in hand shape and/or position during gesturing. Correlation coefficient and structural similarity are the two metrics used for the purpose. Next, several hand parameters such as the finger joint angles and orientation of the palm in these key frames are derived using Google MediaPipe holistic model. When using the app, for a query sign, the corresponding gesture is animated by synthesizing the key frames containing 3D hand shapes with desired palm orientation and finger joint angles using the stored hand parameters, followed by interpolating the in-between frames using image metamorphosis. For interpolation, we have used spline interpolation and intensity interpolation. Some sign gestures created in our experiments show that our proposed method generates smooth and natural-looking animated videos from a comparatively less amount of stored gesture information, thereby offering large savings in memory.
- 24 November 2024
- Published by Springer Nature in Cureus Journal of Computer Science
- Vol. 16  (11)
Open AccessDevelopment of an Animation-Based Indian Sign Language e-Learning App for Persons With Hearing and Speech Impairment
Human-computer interaction (HCI) is changing the quality of our lives. In this paper, we present one such application of HCI that comes as a boon in the life of a hearing-impaired person. We propose to build an e-learning tool for the education of the hearing- and speech-impaired community by creating animated gestures in sign language, with particular emphasis on the Indian Sign Language (ISL). With the help of MATLAB App Designer, we developed a standalone MATLAB application for ISL fingerspelling (including digits, alphabets and combination of these two) and some ISL words of daily usage. This app can be used by the hearing-impaired community for learning ISL, even while seated at home, or for training signers and teachers at special schools. In our proposed method, the software first learns different sign gestures from real gesture videos. For this, hand regions in every frame are extracted using image processing algorithms and subsequently identifying key video frames that show significant change in hand shape and/or position during gesturing. Correlation coefficient and structural similarity are the two metrics used for the purpose. Next, several hand parameters such as the finger joint angles and orientation of the palm in these key frames are derived using Google MediaPipe holistic model. When using the app, for a query sign, the corresponding gesture is animated by synthesizing the key frames containing 3D hand shapes with desired palm orientation and finger joint angles using the stored hand parameters, followed by interpolating the in-between frames using image metamorphosis. For interpolation, we have used spline interpolation and intensity interpolation. Some sign gestures created in our experiments show that our proposed method generates smooth and natural-looking animated videos from a comparatively less amount of stored gesture information, thereby offering large savings in memory.
- Open Access
- 14 September 2024
- Published by ECTI in ECTI Transactions on Computer and Information Technology (ECTI-CIT)
- Vol. 18  (4) , 508-521
BCS-Based Encoding Schemes for Monoview and Multiview Visual Configurations in WVSN Data Gathering: A Survey
Wireless Visual Sensor Network (WVSN) has become a valuable tool in addressing the evolving needs of modern monitoring systems. Encoding in WVSNs is a multifaceted process that involves compressing visual data, optimizing energy consumption, ensuring error resilience, and adapting to various network and application requirements. The associated lightweight encoders and the demand for less storage space make block compressive sensing (BCS) techniques suitable for WVSN applications where energy, bandwidth, and storage resources are limited. Based on the number of visual perspectives or camera angles available within a network for data capture, there are two primary congurations: monoview and multiview. This paper provides a comprehensive survey of dierent BCS-based encoding schemes used for data-gathering in both monoview and multiview scenarios within WVSNs. A comparative study of these algorithms based on compression level, computational complexity, relative gain in encoder energy, and reconstruction quality is performed. A BCS-based joint encoding scheme for multiview conguration that ensures a relatively high compression level is also proposed in this paper.
- 14 September 2024
- Published by ECTI in ECTI Transactions on Computer and Information Technology (ECTI-CIT)
- Vol. 18  (4) , 508-521
Open AccessBCS-Based Encoding Schemes for Monoview and Multiview Visual Configurations in WVSN Data Gathering: A Survey
Wireless Visual Sensor Network (WVSN) has become a valuable tool in addressing the evolving needs of modern monitoring systems. Encoding in WVSNs is a multifaceted process that involves compressing visual data, optimizing energy consumption, ensuring error resilience, and adapting to various network and application requirements. The associated lightweight encoders and the demand for less storage space make block compressive sensing (BCS) techniques suitable for WVSN applications where energy, bandwidth, and storage resources are limited. Based on the number of visual perspectives or camera angles available within a network for data capture, there are two primary congurations: monoview and multiview. This paper provides a comprehensive survey of dierent BCS-based encoding schemes used for data-gathering in both monoview and multiview scenarios within WVSNs. A comparative study of these algorithms based on compression level, computational complexity, relative gain in encoder energy, and reconstruction quality is performed. A BCS-based joint encoding scheme for multiview conguration that ensures a relatively high compression level is also proposed in this paper.
- Open Access
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.
Open AccessMVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.
- 27 June 2024
- Published by Springer Nature in Machine Vision and Applications
- Vol. 35  (4) , 1-15
A multi-modal framework for continuous and isolated hand gesture recognition utilizing movement epenthesis detection
- 27 June 2024
- Published by Springer Nature in Machine Vision and Applications
- Vol. 35  (4) , 1-15
A multi-modal framework for continuous and isolated hand gesture recognition utilizing movement epenthesis detection
- Open Access
GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition
Transformer model have achieved state-of-the-art results in many applications like NLP, classification, etc. But their exploration in gesture recognition task is still limited. So, we propose a novel GestFormer architecture for dynamic hand gesture recognition. The motivation behind this design is to propose a resource efficient transformer model, since transformers are computationally expensive and very complex. So, we propose to use a pooling based token mixer named PoolFormer, since it uses only pooling layer which is a non-parametric layer instead of quadratic attention. The proposed model also leverages the space-invariant features of the wavelet transform and also the multiscale features are selected using multi-scale pooling. Further, a gated mechanism helps to focus on fine details of the gesture with the contextual information. This enhances the performance of the proposed model compared to the traditional transformer with fewer parameters, when evaluated on dynamic hand gesture datasets, NVidia Dynamic Hand Gesture and Briareo datasets. To prove the efficacy of the proposed model, we have experimented on single as well multimodal inputs such as infrared, normals, depth, optical flow and color images. We have also compared the proposed GestFormer in terms of resource efficiency and number of operations. The source code is available at https://github.com/mallikagarg/GestFormer.
Open AccessGestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition
Transformer model have achieved state-of-the-art results in many applications like NLP, classification, etc. But their exploration in gesture recognition task is still limited. So, we propose a novel GestFormer architecture for dynamic hand gesture recognition. The motivation behind this design is to propose a resource efficient transformer model, since transformers are computationally expensive and very complex. So, we propose to use a pooling based token mixer named PoolFormer, since it uses only pooling layer which is a non-parametric layer instead of quadratic attention. The proposed model also leverages the space-invariant features of the wavelet transform and also the multiscale features are selected using multi-scale pooling. Further, a gated mechanism helps to focus on fine details of the gesture with the contextual information. This enhances the performance of the proposed model compared to the traditional transformer with fewer parameters, when evaluated on dynamic hand gesture datasets, NVidia Dynamic Hand Gesture and Briareo datasets. To prove the efficacy of the proposed model, we have experimented on single as well multimodal inputs such as infrared, normals, depth, optical flow and color images. We have also compared the proposed GestFormer in terms of resource efficiency and number of operations. The source code is available at https://github.com/mallikagarg/GestFormer.