Muhammad Shoib Amin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Muhammad Shoib Amin has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Abdul Jabbar
Department of Computer Science, Zhejiang University, Hangzhou, China
3 shared publications - Changbo Wang
East China Normal University, Shanghai, China
3 shared publications - Summaira Jabeen
Zhejiang University, Hangzhou, China
3 shared publications - Summaira Jabeen
College of Computer Science and Technology, Zhejiang University, Hangzhou, China
3 shared publications - Hassan Ali Khan
Software Engineering Institute, East China Normal University, Shanghai, China
2 shared publications - Xi Li
Zhejiang University, Hangzhou, China
2 shared publications
Latest Publications
- Open Access
- 1 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
Dual-Branch Neural Network for Bridging Semantic Gap in Harmful Meme Detection
Memes, cultural entities that spread predominantly through social media, have recently seen an influx of harmful content used for trolling and bullying. Detecting such harmful memes presents a significant challenge due to the semantic disparity between their textual and visual components. To address this issue, we propose a Dual-Branch Neural Network (DBNN) for bridging the semantic gap in harmful meme detection. The proposed DBNN framework utilizes global visual and textual embeddings to learn discriminative joint representations. By capturing the context between the textual and visual modalities, DBNN bridges the semantic gap, enabling robust detection of harmful content.We evaluate the performance of the proposed DBNN framework on the Harm-C and Harm-P benchmark datasets, employing Accuracy and Macro-Averaged Mean Absolute Error (MMAE) as evaluation metrics. Our experiments demonstrate that DBNN significantly outperforms state-of-the-art approaches, achieving higher accuracy and robustness in detecting harmful memes. This work highlights the potential of DBNN in effectively identifying harmful memes, contributing to a safer online environment.
- 1 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
Open AccessDual-Branch Neural Network for Bridging Semantic Gap in Harmful Meme Detection
Memes, cultural entities that spread predominantly through social media, have recently seen an influx of harmful content used for trolling and bullying. Detecting such harmful memes presents a significant challenge due to the semantic disparity between their textual and visual components. To address this issue, we propose a Dual-Branch Neural Network (DBNN) for bridging the semantic gap in harmful meme detection. The proposed DBNN framework utilizes global visual and textual embeddings to learn discriminative joint representations. By capturing the context between the textual and visual modalities, DBNN bridges the semantic gap, enabling robust detection of harmful content.We evaluate the performance of the proposed DBNN framework on the Harm-C and Harm-P benchmark datasets, employing Accuracy and Macro-Averaged Mean Absolute Error (MMAE) as evaluation metrics. Our experiments demonstrate that DBNN significantly outperforms state-of-the-art approaches, achieving higher accuracy and robustness in detecting harmful memes. This work highlights the potential of DBNN in effectively identifying harmful memes, contributing to a safer online environment.
- Open Access
- 17 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 123031-123040
Dual-Attention-Based Enhanced Unified Net for Precise GTV Segmentation of Nasopharyngeal Carcinoma in 3D MR Images
Accurate gross tumor volume (GTV) segmentation is essential for effective radiotherapy in nasopharyngeal carcinoma (NPC). However, challenges arise due to the nasopharyngeal region’s complex anatomy and the annotated data scarcity. Our study presents a dual-attention-based enhanced unified network (DAEU-Net) designed for precise NPC GTV segmentation utilizing 3D T1, T2, and T1C-weighted MR images. Our approach involves splitting large-scale MR data into multiple patches and then training every patch independently. This approach effectively captures localized and detailed information without downscaling the image resolution. The DAEU-Net integrates channel-attention and pixel-attention modules within the encoder section, eliminating background noise and reducing information loss by enhancing the network’s focus on detailed features. The decoder section incorporates bottleneck residual blocks to enhance the computing efficiency and robustness of the network. The proposed methodology surpasses the state-of-the-art models with a respective average symmetric surface distance (ASSD) of 0.920±0.386 mm, 0.987±0.421 mm, and 1.043±0.457 mm and a dice similarity coefficient (DSC) of 0.896, 0.871, and 0.851, respectively. Multi-viewed animated MR images in three orthogonal dimensions (axial, sagittal, and coronal) with predicted NPC tumors and real GTV masks were shown to assist in comprehending the tumor’s precise location. Our approach can significantly improves NPC tumor delineation, aids in automated tumor lesion segmentation, and reduces the annotation workload for oncologists.
- 17 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 123031-123040
Open AccessDual-Attention-Based Enhanced Unified Net for Precise GTV Segmentation of Nasopharyngeal Carcinoma in 3D MR Images
Accurate gross tumor volume (GTV) segmentation is essential for effective radiotherapy in nasopharyngeal carcinoma (NPC). However, challenges arise due to the nasopharyngeal region’s complex anatomy and the annotated data scarcity. Our study presents a dual-attention-based enhanced unified network (DAEU-Net) designed for precise NPC GTV segmentation utilizing 3D T1, T2, and T1C-weighted MR images. Our approach involves splitting large-scale MR data into multiple patches and then training every patch independently. This approach effectively captures localized and detailed information without downscaling the image resolution. The DAEU-Net integrates channel-attention and pixel-attention modules within the encoder section, eliminating background noise and reducing information loss by enhancing the network’s focus on detailed features. The decoder section incorporates bottleneck residual blocks to enhance the computing efficiency and robustness of the network. The proposed methodology surpasses the state-of-the-art models with a respective average symmetric surface distance (ASSD) of 0.920±0.386 mm, 0.987±0.421 mm, and 1.043±0.457 mm and a dice similarity coefficient (DSC) of 0.896, 0.871, and 0.851, respectively. Multi-viewed animated MR images in three orthogonal dimensions (axial, sagittal, and coronal) with predicted NPC tumors and real GTV masks were shown to assist in comprehending the tumor’s precise location. Our approach can significantly improves NPC tumor delineation, aids in automated tumor lesion segmentation, and reduces the annotation workload for oncologists.
- Open Access
- 25 July 2024
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 247
Image-to-image translation based face photo de-meshing using GANs
- 25 July 2024
- Published by Elsevier in Computer Vision and Image Understanding
- Vol. 247
Open AccessImage-to-image translation based face photo de-meshing using GANs
- 26 February 2024
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (32) , 78563-78576
Multimodal pre-train then transfer learning approach for speaker recognition
- 26 February 2024
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (32) , 78563-78576
Multimodal pre-train then transfer learning approach for speaker recognition
- Open Access
Content-based jewellery item retrieval using the local region-based histograms
Jewellery item retrieval is regularly used to find what people want on online marketplaces using a sample query reference image. Considering recent developments, due to the simultaneous nature of various jewelry items, various jewelry goods' occlusion in images or visual streams, as well as shape deformation, content-based jewellery item retrieval (CBJIR) still has limitations whenever it pertains to visual searching in the actual world. This article proposed a content-based jewellery item retrieval method using the local region-based histograms in HSV color space. Using five local regions, our novel jewellery classification module extracts the specific feature vectors from the query image. The jewellery classification module is also applied to the jewellery database to extract feature vectors. Finally, the similarity score is matched between the database and query features vectors to retrieve the jewellery items from the database. The proposed method performance is tested on publicly available jewellery item retrieval datasets, i.e. ringFIR and Fashion Product Images dataset. The experimental results demonstrate the dominance of the proposed method over the baseline methods for retrieving desired jewellery products.
Open AccessContent-based jewellery item retrieval using the local region-based histograms
Jewellery item retrieval is regularly used to find what people want on online marketplaces using a sample query reference image. Considering recent developments, due to the simultaneous nature of various jewelry items, various jewelry goods' occlusion in images or visual streams, as well as shape deformation, content-based jewellery item retrieval (CBJIR) still has limitations whenever it pertains to visual searching in the actual world. This article proposed a content-based jewellery item retrieval method using the local region-based histograms in HSV color space. Using five local regions, our novel jewellery classification module extracts the specific feature vectors from the query image. The jewellery classification module is also applied to the jewellery database to extract feature vectors. Finally, the similarity score is matched between the database and query features vectors to retrieve the jewellery items from the database. The proposed method performance is tested on publicly available jewellery item retrieval datasets, i.e. ringFIR and Fashion Product Images dataset. The experimental results demonstrate the dominance of the proposed method over the baseline methods for retrieving desired jewellery products.
- Open Access
- 18 April 2023
- Published by Slovenian Society for Stereology and Quantitative Image Analysis in Image Analysis and Stereology
- Vol. 42 (1) , 25-35
Improved Median Edge Detection (iMED) for Lossless Image Compression
A wide variety of applications are used in lossless image compression models, especially in medical, space, and aerial imaging domains. Predictive coding improves the performance of lossless image compression, which highly relies on entropy error. Lower entropy error results in better image compression. The main focus of this research is to improve the prediction process by minimizing the entropy error. This paper proposes a novel idea for improved Median Edge Detection (iMED) predictor for lossless image compression. MED predictor is improved using k-means clustering and finding the local context of pixels using 20-Dimensional Difference (DDx20) for input images and updates the cluster weights using learning rates (µi) to minimize the prediction errors of pixels. The performance of the proposed predictor is evaluated on the standard grey-scale test images dataset and KODAK image dataset. Results are obtained and compared based on entropy error, bits per pixel (bpp), and computational running time in seconds(s) with the MED, GAP, FLIF, and LBP predictors. The performance of the proposed iMED predictor improves significantly in terms of the entropy error, bpp, and computational running time in seconds(s) after comparison with different state-of-the-art predictors.
- 18 April 2023
- Published by Slovenian Society for Stereology and Quantitative Image Analysis in Image Analysis and Stereology
- Vol. 42 (1) , 25-35
Open AccessImproved Median Edge Detection (iMED) for Lossless Image Compression
A wide variety of applications are used in lossless image compression models, especially in medical, space, and aerial imaging domains. Predictive coding improves the performance of lossless image compression, which highly relies on entropy error. Lower entropy error results in better image compression. The main focus of this research is to improve the prediction process by minimizing the entropy error. This paper proposes a novel idea for improved Median Edge Detection (iMED) predictor for lossless image compression. MED predictor is improved using k-means clustering and finding the local context of pixels using 20-Dimensional Difference (DDx20) for input images and updates the cluster weights using learning rates (µi) to minimize the prediction errors of pixels. The performance of the proposed predictor is evaluated on the standard grey-scale test images dataset and KODAK image dataset. Results are obtained and compared based on entropy error, bits per pixel (bpp), and computational running time in seconds(s) with the MED, GAP, FLIF, and LBP predictors. The performance of the proposed iMED predictor improves significantly in terms of the entropy error, bpp, and computational running time in seconds(s) after comparison with different state-of-the-art predictors.
- Top Cited PapersOpen Access
- 17 February 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 19 (2s) , 1-41
A Review on Methods and Applications in Multimodal Deep Learning
Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This article focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, physiological signals, flow, RGB, pose, depth, mesh, and point cloud. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the past five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Last, main issues are highlighted separately for each domain, along with their possible future research directions.
- 17 February 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 19 (2s) , 1-41
Top Cited PapersOpen AccessA Review on Methods and Applications in Multimodal Deep Learning
Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This article focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, physiological signals, flow, RGB, pose, depth, mesh, and point cloud. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the past five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Last, main issues are highlighted separately for each domain, along with their possible future research directions.
- 7 June 2022
- Published by Springer Nature in The Visual Computer
- Vol. 39 (9) , 3851-3864
Fashion sub-categories and attributes prediction model using deep learning
- 7 June 2022
- Published by Springer Nature in The Visual Computer
- Vol. 39 (9) , 3851-3864
Fashion sub-categories and attributes prediction model using deep learning