CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models
Authors:
- 9 April 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 684-709
Abstract
Large language models (LLMs) for automatic code generation have recently achieved breakthroughs in several programming tasks. Their advances in competition-level programming problems have made them an essential pillar of AI-assisted pair programming, and tools such as GitHub Copilot have emerged as part of the daily programming workflow used by millions of developers. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure. While these models have been extensively evaluated for their ability to produce functionally correct programs, there remains a lack of comprehensive investigations and benchmarks addressing the security aspects of these models.In this work, we propose a method to systematically study the security issues of code language models to assess their susceptibility to generating vulnerable code. To this end, we introduce the first approach to automatically find generated code that contains vulnerabilities in black-box code generation models. This involves proposing a novel few-shot prompting approach. We evaluate the effectiveness of our approach by examining code language models in generating high-risk security weaknesses. Furthermore, we use our method to create a collection of diverse non-secure prompts for various vulnerability scenarios. This dataset serves as a benchmark to evaluate and compare the security weaknesses of code language models.Keywords
All Related Versions
Funding Information
- Ministry of Education
This publication has 30 references indexed in Scilit:
- Is GitHub’s Copilot as bad as humans at introducing vulnerabilities in code?Empirical Software Engineering, 2023
- An Empirical Study of Code Smells in Transformer-based Code Generation Techniques2022
- Fast, Diverse and Accurate Image Captioning Guided by Part-Of-Speech2019
- Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software2016
- On the capability of static code analysis to detect security vulnerabilitiesInformation and Software Technology, 2015
- Static Analysis: A Survey of Techniques and Tools2015
- Test-Driving Static Analysis Tools in Search of C Code Vulnerabilities2011
- Using Static Analysis to Find BugsIEEE Software, 2008
- Static analysis for securityIEEE Security & Privacy, 2004
- May the source be with youWork Study, 2000