ScamFerret: An LLM Agent for Multi-type and Multilingual Scam Website Detection

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Scam websites defraud consumers worldwide across multiple scam types, including fake online shopping, cryptocurrency scams, and investment scams.Existing detection systems require type-specific labeled datasets and feature engineering, limiting coverage as new scam types emerge.We propose ScamFerret, a Large Language Model (LLM)-based agent that classifies URLs as scam or legitimate by autonomously collecting evidence from web content, infrastructure records, and social media, producing a binary classification and a human-readable rationale without per-type training data.We construct the first ground-truth dataset of 4,780 websites covering eleven scam types in English and seven languages for fake online shopping, with balanced scam--legitimate pairs for four major types.We evaluate six LLMs against six baselines and assess reasoning quality using three metrics (logical consistency, evidence-based reasoning, and feature compliance) validated by human annotation.The best-performing configuration, GPT-4.1, achieves 0.968 mean detection rate across eleven English scam types and 0.983 mean accuracy across seven languages.Model scale systematically shapes evidence-gathering behavior and determines the cost-accuracy trade-off, enabling practitioners to choose between high-accuracy and cost-efficient configurations.These results demonstrate that a single LLM-based agent generalizes across scam types and languages without per-type or per-language training.