Abstract
AI-powered Applications (AI-Apps) on platforms such as Hugging Face give anyone online access to pre-trained models. They lower the barrier to adoption, but they also open an unexplored attack surface: AI-Apps are written by untrusted parties, isolated weakly, and frequently misconfigured. We present the first systematic security analysis of AI-Apps across three leading platforms. Mapping the AI-App lifecycle onto established risk taxonomies (e.g., OWASP) yields five threat categories and ten attack vectors, ranging from generic web flaws to platform-level design faults. Our analysis exposes broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Two vectors are architecture-induced: they survive best-practice development because their root cause lies in the platform rather than in any application. We also show how familiar issues, such as world-readable logs, are amplified in this ecosystem. To assess real-world impact, we built the analysis framework Insightor and ran it over more than 970,000 public AI-Apps. From its output we manually confirmed AI-Apps that leak developer credentials through world-readable logs, AI-Apps whose input handling permits arbitrary code execution, and 27 AI-Apps carrying deliberately planted malicious code. We responsibly disclosed all findings to the affected platforms and developers.
Funding Information
  • Young Scientists Fund of the National Natural Science Foundation of China (62402277)
  • Postdoctoral Fellowship Program of CPSF (GZC20231361)
  • Taishan Scholars Program (N\/A)