CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models
Preprint
Abstract
Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Their advances in competition-level programming problems have made them an essential pillar of AI-assisted pair programming, and tools such as GitHub Copilot have emerged as part of the daily programming workflow used by millions of developers. The training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure. While these models have been extensively assessed for their ability to produce functionally correct programs, there remains a lack of comprehensive investigations and benchmarks addressing the security aspects of these models. In this work, we propose a method to systematically study the security issues of code language models to assess their susceptibility to generating vulnerable code. To this end, we introduce the first approach to automatically find generated code that contains vulnerabilities in black-box code generation models. To achieve this, we present an approach to approximate inversion of the black-box code generation models based on few-shot prompting. We evaluate the effectiveness of our approach by examining code language models in generating high-risk security weaknesses. Furthermore, we establish a collection of diverse non-secure prompts for various vulnerability scenarios using our method. This dataset forms a benchmark for evaluating and comparing the security weaknesses in code language models.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Competition-level code generation with AlphaCodeScience, 2022
- An Empirical Study of Code Smells in Transformer-based Code Generation Techniques2022
- Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion2020
- Fast, Diverse and Accurate Image Captioning Guided by Part-Of-Speech2019
- What developers want and need from program analysis: an empirical study2016
- On the capability of static code analysis to detect security vulnerabilitiesInformation and Software Technology, 2015
- Understanding deep image representations by inverting them2015
- Using Static Analysis to Find BugsIEEE Software, 2008
- A Normalized Levenshtein Distance MetricIEEE Transactions on Pattern Analysis and Machine Intelligence, 2007
- Static analysis for securityIEEE Security & Privacy, 2004