Mallika Garg
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Mallika Garg has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Debashis Ghosh
Department of Electronics and Communication Engineering, Indian Institute of Technology (IIT) Roorkee, Roorkee, India
9 shared publications - Pyari Mohan Pradhan
Department of Electronics and Communication Engineering, Indian Institute of Technology Roorkee, Roorkee, India
9 shared publications Ashwani Kumar Aggarwal
Electrical and Instrumentation Engineering Department, Sant Longowal Institute of Engineering and Technology, Longowal, India
1 shared publicationJagpal Singh Ubhi
Electronics and Communication Engineering Department, Sant Longowal Institute of Engineering and Technology, Longowal, India
1 shared publication
Latest Publications
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 10 (1) , 1-4
Multiview Hand Gesture Recognition From Generated Gestures Using Conditional Adversarial Network
Hand gesture recognition systems often struggles when a gesture is occluded by the hand itself or by the other hand. To address this problem, multiview gestures may be used while training a gesture recognition system. Unfortunately, there are very few datasets available that contains different views of every single gesture. In this paper, we propose a method to handle occlusions in single-view sign gestures by generating multiple views from a single-view gesture using conditional multiview gesture synthesis. The generated views help to solve the occlusion problem, thereby enhancing the recognition performance. Additionally, we introduce a vision-based Multiview Hand Gesture Recognition (MHGR) framework that utilizes the generated multiview gestures for gesture recognition. Experiments conducted on the HGM-4 dataset demonstrate that the generated images are of high quality, photo-realistic, and significantly improve the recognition accuracy compared to some other existing methods.
- 11 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Letters
- Vol. 10 (1) , 1-4
Multiview Hand Gesture Recognition From Generated Gestures Using Conditional Adversarial Network
Hand gesture recognition systems often struggles when a gesture is occluded by the hand itself or by the other hand. To address this problem, multiview gestures may be used while training a gesture recognition system. Unfortunately, there are very few datasets available that contains different views of every single gesture. In this paper, we propose a method to handle occlusions in single-view sign gestures by generating multiple views from a single-view gesture using conditional multiview gesture synthesis. The generated views help to solve the occlusion problem, thereby enhancing the recognition performance. Additionally, we introduce a vision-based Multiview Hand Gesture Recognition (MHGR) framework that utilizes the generated multiview gestures for gesture recognition. Experiments conducted on the HGM-4 dataset demonstrate that the generated images are of high quality, photo-realistic, and significantly improve the recognition accuracy compared to some other existing methods.
- 12 May 2025
- Published by Springer Nature
- p. 15-33
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
- 12 May 2025
- Published by Springer Nature
- p. 15-33
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
- Open Access
OccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.
Open AccessOccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.
- Open Access
Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22\% and 99.10\% on NVGesture and Briareo datasets, respectively.
Open AccessMultiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22\% and 99.10\% on NVGesture and Briareo datasets, respectively.
- Open Access
MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.
Open AccessMVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition
In this paper, we introduce a novel Multiscale Video Transformer Network (MVTN) for dynamic hand gesture recognition, since multiscale features can extract features with variable size, pose, and shape of hand which is a challenge in hand gesture recognition. The proposed model incorporates a multiscale feature hierarchy to capture diverse levels of detail and context within hand gestures which enhances the model's ability. This multiscale hierarchy is obtained by extracting different dimensions of attention in different transformer stages with initial stages to model high-resolution features and later stages to model low-resolution features. Our approach also leverages multimodal data, utilizing depth maps, infrared data, and surface normals along with RGB images from NVGesture and Briareo datasets. Experiments show that the proposed MVTN achieves state-of-the-art results with less computational complexity and parameters. The source code is available at https://github.com/mallikagarg/MVTN.
- Open Access
GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition
Transformer model have achieved state-of-the-art results in many applications like NLP, classification, etc. But their exploration in gesture recognition task is still limited. So, we propose a novel GestFormer architecture for dynamic hand gesture recognition. The motivation behind this design is to propose a resource efficient transformer model, since transformers are computationally expensive and very complex. So, we propose to use a pooling based token mixer named PoolFormer, since it uses only pooling layer which is a non-parametric layer instead of quadratic attention. The proposed model also leverages the space-invariant features of the wavelet transform and also the multiscale features are selected using multi-scale pooling. Further, a gated mechanism helps to focus on fine details of the gesture with the contextual information. This enhances the performance of the proposed model compared to the traditional transformer with fewer parameters, when evaluated on dynamic hand gesture datasets, NVidia Dynamic Hand Gesture and Briareo datasets. To prove the efficacy of the proposed model, we have experimented on single as well multimodal inputs such as infrared, normals, depth, optical flow and color images. We have also compared the proposed GestFormer in terms of resource efficiency and number of operations. The source code is available at https://github.com/mallikagarg/GestFormer.
Open AccessGestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition
Transformer model have achieved state-of-the-art results in many applications like NLP, classification, etc. But their exploration in gesture recognition task is still limited. So, we propose a novel GestFormer architecture for dynamic hand gesture recognition. The motivation behind this design is to propose a resource efficient transformer model, since transformers are computationally expensive and very complex. So, we propose to use a pooling based token mixer named PoolFormer, since it uses only pooling layer which is a non-parametric layer instead of quadratic attention. The proposed model also leverages the space-invariant features of the wavelet transform and also the multiscale features are selected using multi-scale pooling. Further, a gated mechanism helps to focus on fine details of the gesture with the contextual information. This enhances the performance of the proposed model compared to the traditional transformer with fewer parameters, when evaluated on dynamic hand gesture datasets, NVidia Dynamic Hand Gesture and Briareo datasets. To prove the efficacy of the proposed model, we have experimented on single as well multimodal inputs such as infrared, normals, depth, optical flow and color images. We have also compared the proposed GestFormer in terms of resource efficiency and number of operations. The source code is available at https://github.com/mallikagarg/GestFormer.
- 2 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 30 (10709908) , 80-84
Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22% and 99.10% on NVGesture and Briareo datasets, respectively.
- 2 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 30 (10709908) , 80-84
Multiscaled Multi-Head Attention-Based Video Transformer Network for Hand Gesture Recognition
Dynamic gesture recognition is one of the challenging research areas due to variations in pose, size, and shape of the signer's hand. In this letter, Multiscaled Multi-Head Attention Video Transformer Network (MsMHA-VTN) for dynamic hand gesture recognition is proposed. A pyramidal hierarchy of multiscale features is extracted using the transformer multiscaled head attention model. The proposed model employs different attention dimensions for each head of the transformer which enables it to provide attention at the multiscale level. Further, in addition to single modality, recognition performance using multiple modalities is examined. Extensive experiments demonstrate the superior performance of the proposed MsMHA-VTN with an overall accuracy of 88.22% and 99.10% on NVGesture and Briareo datasets, respectively.
- 22 September 2022
- Published by Springer Nature
- p. 383-392
A Two-Stage Convolutional Neural Network for Hand Gesture Recognition
- 22 September 2022
- Published by Springer Nature
- p. 383-392
A Two-Stage Convolutional Neural Network for Hand Gesture Recognition
- 8 August 2022
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (4) , 6271-6288
Neural style transfer for image steganography and destylization with supervised image to image translation
- 8 August 2022
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (4) , 6271-6288
Neural style transfer for image steganography and destylization with supervised image to image translation
- 24 July 2022
- Published by Springer Nature
- p. 308-318
Pose Guided Controllable Gesture to Gesture Translation
- 24 July 2022
- Published by Springer Nature
- p. 308-318
Pose Guided Controllable Gesture to Gesture Translation