Your Space is My Zone: Demystifying the Security Risks of AI-Powered Applications on Pre-Trained Model Hubs
Open Access
- 14 November 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 4582-4596
Abstract
AI-powered Applications (AI-Apps) on platforms such as Hugging Face give anyone online access to pre-trained models. They lower the barrier to adoption, but they also open an unexplored attack surface: AI-Apps are written by untrusted parties, isolated weakly, and frequently misconfigured. We present the first systematic security analysis of AI-Apps across three leading platforms. Mapping the AI-App lifecycle onto established risk taxonomies (e.g., OWASP) yields five threat categories and ten attack vectors, ranging from generic web flaws to platform-level design faults. Our analysis exposes broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Two vectors are architecture-induced: they survive best-practice development because their root cause lies in the platform rather than in any application. We also show how familiar issues, such as world-readable logs, are amplified in this ecosystem. To assess real-world impact, we built the analysis framework Insightor and ran it over more than 970,000 public AI-Apps. From its output we manually confirmed AI-Apps that leak developer credentials through world-readable logs, AI-Apps whose input handling permits arbitrary code execution, and 27 AI-Apps carrying deliberately planted malicious code. We responsibly disclosed all findings to the affected platforms and developers.Keywords
Funding Information
- Young Scientists Fund of the National Natural Science Foundation of China (62402277)
- Postdoctoral Fellowship Program of CPSF (GZC20231361)
- Taishan Scholars Program (N\/A)
This publication has 20 references indexed in Scilit:
- My Model is Malware to You: Transforming AI Models into Malware by Abusing TensorFlow APIs2025
- Hey, Your Secrets Leaked! Detecting and Characterizing Secret Leakage in the Wild2025
- SoK: Taxonomy of Attacks on Open-Source Software Supply Chains2023
- Investigating Package Related Security Threats in Software Registries2023
- Security and Privacy Risks of Number Recycling at Mobile Carriers in the United States2021
- Secrets in Source Code: Reducing False Positives using Machine Learning2020
- Characterizing Pixel Tracking through the Lens of Disposable Email Services2019
- Detecting and Mitigating Secret-Key Leaks in Source Code Repositories2015
- Statistical Measurement of Information Leakage2010
- Automatic Discovery and Quantification of Information Leaks2009