Systematically Finding Security Vulnerabilities in Black-Box Code Generation Models
Authors:
Preprint
Abstract
Recently, large language models for code generation have achieved breakthroughs in several programming language tasks. Their advances in competition-level programming problems have made them an emerging pillar in AI-assisted pair programming. Tools such as GitHub Copilot are already part of the daily programming workflow and are used by more than a million developers. The training data for these models is usually collected from open-source repositories (e.g., GitHub) that contain software faults and security vulnerabilities. This unsanitized training data can lead language models to learn these vulnerabilities and propagate them in the code generation procedure. Given the wide use of these models in the daily workflow of developers, it is crucial to study the security aspects of these models systematically. In this work, we propose the first approach to automatically finding security vulnerabilities in black-box code generation models. To achieve this, we propose a novel black-box inversion approach based on few-shot prompting. We evaluate the effectiveness of our approach by examining code generation models in the generation of high-risk security weaknesses. We show that our approach automatically and systematically finds 1000s of security vulnerabilities in various code generation models, including the commercial black-box model GitHub Copilot.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Competition-level code generation with AlphaCodeScience, 2022
- An Empirical Study of Code Smells in Transformer-based Code Generation Techniques2022
- A systematic evaluation of large language models of code2022
- Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions2022
- Fast, Diverse and Accurate Image Captioning Guided by Part-Of-Speech2019
- What developers want and need from program analysis: an empirical study2016
- Understanding deep image representations by inverting them2015
- Test-Driving Static Analysis Tools in Search of C Code Vulnerabilities2011
- Using Static Analysis to Find BugsIEEE Software, 2008
- A Normalized Levenshtein Distance MetricIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007