A Hybrid Deep Learning Approach for Image-based Malware Classification
Open Access
- 28 February 2026
- journal article
- Published by Fırat University, Faculty of Engineering in Firat University Journal of Experimental and Computational Engineering
- Vol. 5 (1) , 26-43
Abstract
The diversity and sophistication of malicious content has significantly impacted end-users of Information and Communication Technologies. In order to mitigate the impact of malicious content, automated deep learning-based techniques have been developed to proactively defend user systems against malware. In this study, we implement a hybrid model for malware detection and classification using the MaleVis dataset. First, feature extraction from the dataset is performed using DenseNet-121, EfficientNet-B0 and ResNet-50 models. These models are deep learning architectures that have been trained on large datasets and are known for their powerful feature extraction capabilities. Each model was used to extract feature vectors from the images in the Malevis dataset. These feature vectors were then merged. The combined feature vectors were used for classification using XGBoost, a powerful classification algorithm. This hybrid model approach combines the feature extraction capabilities of deep learning models with the classification capability of XGBoost to detect malware. Experimental results show that the proposed hybrid model achieves high accuracy rates on the MaleVis dataset. The study shows that combining the feature extraction capabilities of different deep learning models and using these features with a classifier such as XGBoost can provide significant improvements in malware detection and classification.The results demonstrate the model’s potential for integration into real-world threat detection systems.Keywords
This publication has 29 references indexed in Scilit:
- Artificial Intelligence for Cybersecurity: Threats, Attacks and Mitigation2023
- A deep learning approach for classification of COVID and pneumonia using DenseNet‐201International Journal of Imaging Systems and Technology, 2022
- Spoofing detection system for e-health digital twin using EfficientNet Convolution Neural NetworkMultimedia Tools and Applications, 2022
- ResNet 502021
- EfficientNet2021
- Automated malware identification method using image descriptors and singular value decompositionMultimedia Tools and Applications, 2021
- Automated malware recognition method based on local neighborhood binary patternMultimedia Tools and Applications, 2020
- HIT4Mal: Hybrid image transformation for malware classificationTransactions on Emerging Telecommunications Technologies, 2019
- Analysis of ResNet and GoogleNet models for malware detectionJournal of Computer Virology and Hacking Techniques, 2018
- A survey of cyber crimesSecurity and Communication Networks, 2011