Shahryar Baki

Despite two decades of research on automatic filtering systems, phishing attacks remain a serious problem. To alleviate risks from filtering failures, we design and evaluate the effectiveness of a new warning system on users' susceptibility to phishing. Our proposed technique highlights key sentences based on an analysis of the persuasive techniques used. An online mixed-design study ( $n=604$ ) shows that adding our highlighting technique outperforms existing warning solutions. It also identifies the relative efficacy of different appeals and the characteristics of susceptible users. Results show that adding our highlighting technique is useful even with false positives and false negatives . Inspired by this result, we propose an automatic warning generator. We created a small labeled dataset of suspicious sentences and used data augmentation. Our best models achieve F1 score of 99.95% in detecting phishing emails and 88% in detecting suspicious sentences.

Several previous studies have investigated user susceptibility to phishing attacks. A thorough meta-analysis or systematic review is required to gain a better understanding of these findings and to assess the strength of evidence for phishing susceptibility of a subpopulation, e.g., older users. We aim to determine whether an effect exists; another aim is to determine whether the effect is positive or negative and to obtain a single summary estimate of the effect. OBJECTIVES: We systematically review the results of previous user studies on phishing susceptibility and conduct a meta-analysis. METHOD: We searched four online databases for English studies on phishing. We included all user studies in phishing detection and prevention, whether they proposed new training techniques or analyzed users' vulnerability. FINDINGS: A careful analysis reveals some discrepancies between the findings. More than half of the studies that analyzed the effect of age reported no statistically significant relationship between age and users' performance. Some studies reported older people performed better while some reported the opposite. A similar finding holds for the gender difference. The meta-analysis shows: 1) a significant relationship between participants' age and their susceptibility 2) females are more susceptible than males 3) users training significantly improves their detection ability.

We perform an in-depth, systematic benchmarking study and evaluation of phishing features on diverse and extensive datasets. We propose a new taxonomy of features based on the interpretation and purpose of each feature. Next, we propose a benchmarking framework called `PhishBench,' which enables us to evaluate and compare the existing features for phishing detection systematically and thoroughly under identical experimental conditions, i.e., unified system specification, datasets, classifiers, and evaluation metrics. PhishBench is a first in the field of benchmarking phishing related research and incorporates thorough and systematic evaluation and feature comparison. We use PhishBench to test methods published in the phishing literature on new and diverse datasets to check their robustness and scalability. We study how dataset characteristics, e.g., varying legitimate to phishing ratios and increasing the size of imbalanced datasets, affect classification performance. Our results show that the imbalanced nature of phishing attacks affects the detection systems' performance and researchers should take this into account when proposing a new method. We also found that retraining alone is not enough to defeat new attacks. New features and techniques are required to stop attackers from fooling detection systems.

Phishing and spear phishing are typical examples of masquerade attacks since trust is built up through impersonation for the attack to succeed. Given the prevalence of these attacks, considerable research has been conducted on these problems along multiple dimensions. We reexamine the existing research on phishing and spear phishing from the perspective of the unique needs of the security domain, which we call security challenges: real-time detection, active attacker, dataset quality and base-rate fallacy. We explain these challenges and then survey the existing phishing/spear phishing solutions in their light. This viewpoint consolidates the literature and illuminates several opportunities for improving existing solutions. We organize the existing literature based on detection techniques for different attack vectors (e.g., URLs, websites, emails) along with studies on user awareness. For detection techniques we examine properties of the dataset, feature extraction, detection algorithms used, and performance evaluation metrics. This work can help guide the development of more effective defenses for phishing, spear phishing and email masquerade attacks of the future, as well as provide a framework for a thorough evaluation and comparison.