Abstract
With the proliferation of web technologies, web servers have become high-value targets for cyberattacks. A key attack vector involves the deployment of WebShells—malicious scripts that enable unauthorized remote access. Despite growing research on WebShell detection, most studies focus on novel methods and offer limited evaluations against existing open-source tools. To the best of our knowledge, this work presents the first systematic benchmarking and analysis of open-source WebShell detectors. We evaluate both static and dynamic detection tools using a dataset of 2,188 diverse PHPWebShells, examining tool performance across accuracy, false positives, and runtime efficiency. Furthermore, we propose a novel static detection approach that leverages Large Language Models (LLMs) to automatically generate regex-based detection rules. Our LLM-generated ruleset achieves the highest detection rate among all evaluated static methods at 92%, while maintaining an exceptionally low false positive rate of 0.49%. These results demonstrate the potential of LLMs as an effective aid in crafting high-quality detection signatures. Through our comprehensive evaluation, we provide actionable insights and recommendations tailored to real-world usage scenarios, empowering developers and security practitioners to select and build appropriate detection solutions based on practical constraints and goals.