PhreshPhish: A Real-World, High-Quality, Large-Scale Phishing Website Dataset and Benchmark
Authors:
- Thomas DaltonTDThomas Dalton
- Hemanth GowdaHGHemanth Gowda
- Girish RaoGRGirish Rao
- Sachin PargiSPSachin Pargi
- Alireza Hadj KhodabakhshiAKAlireza Hadj Khodabakhshi
- Joseph RombsJRJoseph Rombs
- Stephan JouSJStephan Jou
- Manish MarwahMMManish Marwah
Preprint
Abstract
Phishing remains a pervasive and growing threat, inflicting heavy economic and reputational damage. While machine learning has been effective in real-time detection of phishing attacks, progress is hindered by lack of large, high-quality datasets and benchmarks. In addition to poor-quality due to challenges in data collection, existing datasets suffer from leakage and unrealistic base rates, leading to overly optimistic performance results. In this paper, we introduce PhreshPhish, a large-scale, high-quality dataset of phishing websites that addresses these limitations. Compared to existing public datasets, PhreshPhish is substantially larger and provides significantly higher quality, as measured by the estimated rate of invalid or mislabeled data points. Additionally, we propose a comprehensive suite of benchmark datasets specifically designed for realistic model evaluation by minimizing leakage, increasing task difficulty, enhancing dataset diversity, and adjustment of base rates more likely to be seen in the real world. We train and evaluate multiple solution approaches to provide baseline performance on the benchmark sets. We believe the availability of this dataset and benchmarks will enable realistic, standardized model comparison and foster further advances in phishing detection. The datasets and benchmarks are available on Hugging Face (https://huggingface.co/datasets/phreshphish/phreshphish).Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- A scalable phishing website detection model based on dual-branch TCN and mask attentionComputer Networks, 2025
- Benchmarking and Evaluating Large Language Models in Phishing Detection for Small and Midsize Enterprises: A Comprehensive AnalysisIEEE Access, 2025
- ChatPhishDetector: Detecting Phishing Sites Using Large Language ModelsIEEE Access, 2024
- A State-of-the-Art Review on Phishing Website Detection TechniquesIEEE Access, 2024
- Phishing or Not Phishing? A Survey on the Detection of Phishing WebsitesIEEE Access, 2023
- An effective detection approach for phishing websites using URL and HTML featuresScientific Reports, 2022
- A machine learning approach for detecting fast flux phishing hostnamesJournal of Information Security and Applications, 2022
- An In-Depth Benchmarking and Evaluation of Phishing Detection Research for Security NeedsIEEE Access, 2020
- SoK: A Comprehensive Reexamination of Phishing Research From the Security PerspectiveIEEE Communications Surveys & Tutorials, 2019
- Near-optimal hashing algorithms for approximate nearest neighbor in high dimensionsCommunications of the ACM, 2008