Cross-Corpus Evaluation of Classical Machine Learning Models for Spam Email Detection

Authors:
Open Access
Publisher website
Google Scholar
Abstract
In today’s world of rapidly evolving internet communication, email remains a widely used medium for direct message exchange across ordinary users, organizations, and defense-related environments, where spam campaigns can threaten operational continuity and communication integrity. Despite significant advances in spam filtering techniques using deep learning approaches, comparatively less attention has been given to the robustness of traditional machine learning classifiers under cross-corpus distribution shift. This study benchmarks five classical classifiers—Naive Bayes, Logistic Regression, Random Forest, Linear Support Vector Machine (SVM), and XGBoost—for spam email detection. A unified framework incorporating text preprocessing, TF-IDF feature extraction, engineered metadata features, stratified 10-fold cross-validation, and cross-corpus evaluation was applied to the sampled public email corpora, including one LLM-generated synthetic corpus (Dataset C). Classifier performance was assessed using standard classification metrics, dependence-aware statistical testing, retention-based analysis, and confusion-matrix evaluation. Corpus characteristics were examined using lexical diversity, vocabulary overlap, TF-IDF sparsity, and metadata distribution analyses. Results showed that Linear SVM achieved the highest within-corpus F1-score in the sampled conventional corpora (0.9868 on Dataset A), but this ranking did not consistently carry over to the tested transfers, where performance varied by classifier, feature set, and direction. XGBoost with combined features achieved an F1-score of 0.9233 in the A-to-C transfer, while Linear SVM achieved 0.9437 in B-to-C; these results do not establish general model rankings. Metadata-only scores above the within-corpus baseline in some transfers were consistent with the target class composition, including all-spam predictions, and do not establish a transferable metadata effect. The findings are limited to the sampled corpora and experimental settings; transfer to Dataset C does not establish generalization to LLM-generated spam more broadly.

This publication has 25 references indexed in Scilit: